Control device, picking system, distribution system, program, and control method

ABSTRACT

A point cloud information acquiring unit for acquiring point cloud information of a region including at least one of the picking hand and the work at a time which is at least one of (i) when the picking hand picks the work at the first position, (ii) when the picking hand moves the work out of the first container, (iii) when the picking hand moves the work in the second container and (iv) when the picking hand arranges the work at the second position; and an abnormality detection unit for detecting an abnormality based on the point cloud information acquired by the point cloud information acquiring unit are included.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/JP2017/014151 filed on Apr. 4, 2017, the contents of which arehereby incorporated herein by reference in its entirety.

BACKGROUND Technical Field

The subject disclosure relates to a control device, a picking system, adistribution system, a program and a control method.

Background

A picking system has been known, in which a picking hand and a robot armare used to take out a work stored in a container and arrange the workat a specific position within another container.

CITATION LIST Patent Literature

[PTL 1]

Japanese Patent Application Publication No. 2016-091053

[PTL 2]

Japanese Patent Application Publication No. 2016-147330

[PTL 3]

Japanese Patent Application Publication No. 2017-033429

SUMMARY

In a case of automatically calculating a trajectory of a work by takingan interference with an obstacle into account, there are numerousamounts of calculation and it is difficult to calculate a trajectorywithin practical time.

General Disclosure

In a first aspect of the present invention, a control device isprovided. The above-described control device may be a control device forcontrolling a picking apparatus. In the above-described control device,the picking apparatus may arrange, at a second position of a secondcontainer, a work picked at a first position of a first container byusing a picking hand, the second container being different from thefirst container. The above-described control device includes a pointcloud information acquiring unit for acquiring point cloud informationof a region including at least one of the picking hand and the work at atime which is at least one of (i) when a picking hand picks a work at afirst position, (ii) when the picking hand moves the work out of a firstcontainer, (iii) when the picking hand moves the work in a secondcontainer and (iv) when the picking hand arranges the work at a secondposition, for example. The above-described control device includes anabnormality detection unit for detecting an abnormality based on pointcloud information acquired by the point cloud information acquiringunit, for example.

The above-described control device may include an image acquiring unitfor acquiring image data of an image including at least one of thepicking hand and the work in a field angle at the at least one time. Inthe above-described control device, the point cloud informationacquiring unit may acquire the point cloud information based on theimage acquired by the image acquiring unit. In the above-describedcontrol device, the point cloud information may be three-dimensionalpoint cloud information.

In the above-described control device, the abnormality detection unitmay detect the abnormality based on (i) the point cloud information and(ii) at least one selected from a group including a measurement valuefor a sensor arranged on the picking apparatus, a control value for atleast one of the picking hand, and a robot arm coupled with the pickinghand, and at least one of torque, a voltage value and a current valuefor a driving unit that drives at least one of the picking hand and therobot arm. The above-described control device may include an instructionsending unit for sending, to the picking apparatus, an instruction forsuspending the operation of the picking apparatus when the abnormalitydetection unit for detecting the abnormality.

In a second aspect of the present invention, a picking system isprovided. The above-described picking system includes theabove-described control device, for example. The above-described pickingsystem includes the above-described picking apparatus, for example.

In a third aspect of the present invention, a distribution system isprovided. The above-described distribution system includes the controldevice, for example. The above-described distribution system includesthe picking apparatus, for example. The above-described distributionsystem includes a conveying apparatus to convey the work, for example.

In a fourth aspect of the present invention, a program is provided. Anon-transitory computer-readable medium to store the above-describedprogram may also be provided. The above-described program may be aprogram to cause a computer to serve as the above-described controldevice.

In a fourth aspect of the present invention, a control method isprovided. The above-described control method may be a control method forcontrolling a picking apparatus. The above-described control method maybe performed by a computer. In the above-described control method, thepicking apparatus may arrange, at a second position of a secondcontainer, a work picked at a first position of a first container byusing a picking hand, the second container being different from thefirst container. The above-described control method includes a pointcloud information acquiring procedure of acquiring point cloudinformation of a region including at least one of the picking hand andthe work at a time which is at least one of (i) when the picking handpicks a work at a first position, (ii) when the picking hand moves thework out of a first container, (iii) when the picking hand moves thework in a second container, and (iv) when the picking hand arranges thework at a second position, for example. The above-described controlmethod includes an abnormality detection procedure of detecting theabnormality based on the point cloud information acquired in the pointcloud information acquiring procedure, for example.

The summary clause does not necessarily describe all necessary featuresof the embodiments of the present invention. The present invention mayalso be a sub-combination of the features described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows one example of a system configuration of adistribution management system 100.

FIG. 2 schematically shows one example of an internal configuration ofan article management apparatus 110.

FIG. 3 schematically shows one example of an internal configuration of adata registration apparatus 120.

FIG. 4 schematically shows one example of an internal configuration of arobot 140.

FIG. 5 schematically shows one example of an internal configuration of atransfer control apparatus 160.

FIG. 6 schematically shows one example of an internal configuration of arecognition unit 520.

FIG. 7 schematically shows one example of an internal configuration of aplanning unit 530.

FIG. 8 schematically shows another example of an internal configurationof an execution unit 540.

FIG. 9 schematically shows one example of information processing in atransfer control apparatus 160.

FIG. 10 schematically shows one example of the recognition processing inS912.

FIG. 11 schematically shows one example of planning processing n S914.

FIG. 12 schematically shows one example of trajectory planningprocessing in S1160.

FIG. 13 schematically shows one example of interference checkingprocessing in S1208.

FIG. 14 schematically shows one example of a data table 1400.

FIG. 15 schematically shows one example of concurrent processing betweenworks.

FIG. 16 schematically shows one example of concurrent processing betweenworks.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, (some) embodiment(s) of the present invention will bedescribed. The embodiment(s) do(es) not limit the invention according tothe claims, and all the combinations of the features described in theembodiment(s) are not necessarily essential to means provided by aspectsof the invention. Note that in the drawings, the same or similarportions are provided with the same reference signs and redundantdescriptions may be omitted.

[Summary of Distribution Management System 100]

FIG. 1 schematically shows one example of a system configuration of adistribution management system 100. In the present embodiment, thedistribution management system 100 includes an article managementapparatus 110, a data registration apparatus 120, a transfer apparatus130 and a conveying apparatus 170. In the present embodiment, thetransfer apparatus 130 includes a robot 140, an image capturingapparatus 152, an image capturing apparatus 154 and a transfer controlapparatus 160. In the present embodiment, the conveying apparatus 170includes a conveyor 172 that conveys the supply pallet 12 and a conveyor174 that conveys a delivery pallet 14.

The supply pallet 12 may be one example of a first container and asupply container. The delivery pallet 14 may be one example of a secondcontainer and an arrangement container. The distribution managementsystem 100 may be one example of a control device, an informationprocessing apparatus, a picking system and a distribution system. Thearticle management apparatus 110 may be one example of a storageapparatus. The data registration apparatus 120 may be one example of theinformation processing apparatus. The transfer apparatus 130 may be oneexample of the picking apparatus and the picking system. The robot 140may be one example of the picking apparatus. The image capturingapparatus 152 may be one example of an image acquiring unit. The imagecapturing apparatus 154 may be one example of the image acquiring unit.The transfer control apparatus 160 may be one example of the controldevice and the picking control unit.

In the present embodiment, the distribution management system 100manages warehousing, storage and shipping of a plurality of types ofarticles. For example, the distribution management system 100 produces apiece of baggage for delivery by putting a stored article in a containerfor delivery based on an order from a user.

The baggage for delivery is produced according to the followingprocedures, for example. First, the conveyor 172 of the conveyingapparatus 170 conveys the supply pallet 12, in which an article orderedby a user is stored, to a working position of the transfer apparatus130. In one embodiment, the supply pallet 12 stores one article. Inanother embodiment, the supply pallet 12 stores a plurality of articlesof the same type. In still another embodiment, the supply pallet 12stores a plurality of types of articles. Also, the conveyor 174 of theconveying apparatus 170 conveys the delivery pallet 14 to a workingposition of the transfer apparatus 130.

Next, the transfer apparatus 130 identifies a work 10 that is to be aworking target of the transfer apparatus 130 from among the articlesstored in the supply pallet 12. Also, the transfer apparatus 130 sets aspecific position or any position of the delivery pallet 14 as anarrangement position of the above-described work 10. The transferapparatus 130 determines a trajectory for moving the work 10 of thesupply pallet 12 to the specific position or any position of thedelivery pallet 14. The transfer apparatus 130 (i) picks and takes theidentified work 10 out of the supply pallet 12, and (ii) arranges theidentified work 10 to the specific position or any position of thedelivery pallet 14.

When another article ordered by the user has been stored in theabove-described supply pallet 12, the transfer apparatus 130 repeats asimilar working to the above. When the other article ordered by the userhas been stored in another supply pallet 12, the conveyor 172 moves theabove-described supply pallet 12 out of the working position of thetransfer apparatus 130, and conveys the above-described other supplypallet 12 to the working position of the transfer apparatus 130. Thetransfer apparatus 130 repeats a similar working to the above withrespect to another article stored in another supply pallet 12. Then, asall articles ordered by the user are stored in the delivery pallet 14,the conveyor 174 moves the delivery pallet 14 out of the workingposition of the transfer apparatus 130. Accordingly, pieces of baggagefor delivery can be prepared.

[Summary of Each Unit of Distribution Management System 100]

In the present embodiment, the article management apparatus 110 manageswarehousing, storage and shipping of articles. The article managementapparatus 110 acquires user order information that indicates ordercontents of a user. Details of the article management apparatus 110 aredescribed below.

In the present embodiment, the data registration apparatus 120 acquiresvarious information regarding warehoused articles. For example, the dataregistration apparatus 120 acquires information regarding appearances,shapes, specifications, handling methods and the like. The dataregistration apparatus 120 may register the acquired information in thearticle management apparatus 110. The data registration apparatus 120may also register information about the all warehoused articles, and mayalso register information per type of articles. Details of the dataregistration apparatus 120 are described below.

In the present embodiment, the transfer apparatus 130 moves the work 10from the supply pallet 12 arranged at the working position of thetransfer apparatus 130 to the delivery pallet 14 arranged at the workingposition of the transfer apparatus 130. To simplify the description, inthe present embodiment, a case in which a single supply pallet 12 and asingle delivery pallet 14 are arranged at the working positions of thetransfer apparatus 130 is described. However, the transfer apparatus 130is not limited to the present embodiment.

In another embodiment, a plurality of supply pallets 12 and one or moredelivery pallets 14 are arranged at working positions of the transferapparatus 130, and the transfer apparatus 130 moves a plurality of worksbetween the plurality of supply pallets 12 and the one or more deliverypallets 14. In still another embodiment, one or more supply pallets 12and a plurality of delivery pallets 14 are arranged at the workingpositions of the transfer apparatus 130, and the transfer apparatus 130moves a plurality of works between the one or more supply pallets 12 andthe plurality of delivery pallets 14.

In the present embodiment, the robot 140 moves the work 10 from thesupply pallet 12 to the delivery pallet 14 based on an instruction fromthe transfer control apparatus 160. For example, the robot 140 moves, byusing the picking hand, one or more works 10 from one or more supplypallets 12 to one or more delivery pallets 14. The robot 140 may alsoarrange, at a second position, the work 10 picked at a first position byusing the picking hand, the second position being different from thefirst position. The first position may be a specific position or anyposition of the supply pallet 12. The second position may be a specificposition or any position of the delivery pallet 14. Details of the robot140 are described below. Note that the expression “one or more” may bereferred to as the expression “one or more”.

In the present embodiment, the image capturing apparatus 152 captures animage of the supply pallet 12 arranged at the working position of thetransfer apparatus 130. The image capturing apparatus 152 may alsoacquire three-dimensional information (for example, a three-dimensionalpoint cloud) of a surface of the supply pallet 12. The image capturingapparatus 152 may also capture a stereo image of the supply pallet 12.The image capturing apparatus 152 may also acquire temperatureinformation of the supply pallet 12. As the image capturing apparatus152, a camera, a stereo camera, a three-dimensional sensor and the likecan be illustrated. The image capturing apparatus 152 may also include aplurality of image capturing apparatuses.

In the present embodiment, the image capturing apparatus 154 captures animage of the delivery pallet 14 arranged at the working position of thetransfer apparatus 130. The image capturing apparatus 154 may alsoacquire three-dimensional information (for example, a three-dimensionalpoint cloud) of a surface of the delivery pallet 14. The image capturingapparatus 152 may also capture a stereo image of the supply pallet 12.The image capturing apparatus 154 may also acquire temperatureinformation of the delivery pallet 14. As the image capturing apparatus154, a camera, a stereo camera, a three-dimensional sensor and the likecan be illustrated. The image capturing apparatus 154 may also include aplurality of image capturing apparatuses.

In the present embodiment, the transfer control apparatus 160 controlsthe transfer apparatus 130. In one embodiment, the transfer controlapparatus 160 is arranged outside the transfer apparatus 130. In anotherembodiment, the transfer control apparatus 160 is arranged inside thetransfer apparatus 130. Details of the transfer control apparatus 160are described below.

In the present embodiment, the conveying apparatus 170 conveys one ormore works. For example, the conveying apparatus 170 conveys one or moreworks stored in the supply pallet 12. The conveying apparatus 170 mayalso convey one or more works stored in the delivery pallet 14.

[Specific Configuration of Each Unit of Distribution Management System100]

Each unit of the distribution management system 100 may also be realizedby hardware, may also be realized by software, and may also be realizedby hardware and software. At least some of the respective units of thedistribution management system 100 may also be realized by a singleserver, or may also be realized by a plurality of servers. At least someof the respective units of the distribution management system 100 mayalso be realized on a virtual server or a cloud system. At least some ofthe respective units of the distribution management system 100 may alsobe realized by a personal computer or a mobile terminal. As the mobileterminal, a mobile phone, a smartphone, a PDA, a tablet, a notebookcomputer or a laptop computer, a wearable computer and the like can beillustrated. Each unit of the distribution management system 100 mayalso store information by using a distributed inventory technology suchas a block chain, or a distributed network.

In a case in which at least some of the components configuring thedistribution management system 100 are realized by software, thecomponent realized by the software may be realized by activatingsoftware or a program prescribing an operation regarding the componentin an information processing apparatus in a general configuration. Theabove-described information processing apparatus in the generalconfiguration may include: (i) a data processing apparatus having aprocessor, such as CPU and GPU, a ROM, a RAM, a communication interfaceand the like; (ii) an input apparatus such as a keyboard, a pointingdevice, a touch panel, a camera, a sound input apparatus, a gestureinput apparatus, various sensors and a GPS receiver; (iii) an outputapparatus such as a display apparatus, a sound output apparatus and avibration apparatus; and (iv) a storage apparatus (including an externalstorage apparatus) such as a memory, HDD and SSD.

In the above-described information processing apparatus in the generalconfiguration, the above-described data processing apparatus or storageapparatus may store the above-described software or program. Theabove-described software or program is performed by the processor, andaccordingly, causes the above-described information processing apparatusto perform an operation prescribed by the software or program. Theabove-described software or program may also be stored in anon-transitory computer-readable medium. The above-described software orprogram may be a program to cause a computer to serve as the transfercontrol apparatus 160 or a part of the transfer control apparatus 160.

The above-described software or program may be a program to control thepicking apparatus. In the above-described program, the picking apparatusmay arrange, at a second position, a work picked at a first position byusing a picking hand, the second position being different from theabove-described first position. The above-described program may be aprogram to cause a computer to perform a first candidate informationacquiring procedure of acquiring first candidate information indicatingeach of one or more postures that the above-described picking hand cantake at the above-described first position. The above-described programmay be a program to cause a computer to perform a second candidateinformation acquiring procedure of acquiring second candidateinformation indicating each of one or more postures that theabove-described picking hand can take at the above-described secondposition, and an extracting procedure of extracting one combination fromamong one or more combinations regarding the one or more posturesindicated by the above-described first candidate information and the oneor more postures indicated by the above-described second candidateinformation. The above-described program may be a program to cause acomputer to perform a trajectory calculating procedure of calculating atrajectory on which (i) the above-described picking hand picks, at theabove-described first position, the above-described work with a postureassociated with the above-described first position included in theabove-described one combination, and (ii) the above-described pickinghand arranges, at the above-described second position, theabove-described work with a posture associated with the above-describedsecond position included in the above-described one combination. Theabove-described program may be a program to cause a computer to performa determining procedure of determining whether an interference ispresent or absent on the trajectory that is calculated in theabove-described trajectory calculating procedure.

The above-described software or program may be a program to control thepicking apparatus. In the above-described program, the picking apparatusmay arrange, at a second position of a second container, a work pickedat a first position of a first container by using a picking hand, thesecond container being different from the above-described firstcontainer. The above-described program may be a program to cause acomputer to perform a trajectory information generating procedure ofgenerating trajectory information prescribing a trajectory on which (i)the above-described picking hand picks, at the above-described firstposition, the above-described work with a posture associated with theabove-described first position included in the above-described onecombination, and (ii) the picking hand arranges, at the above-describedsecond position, the above-described work with a posture associated withthe above-described second position included in the above-described onecombination. The above-described program may be a program to cause acomputer to perform an execution control procedure to operate theabove-described picking apparatus based on the trajectory informationgenerated in the above-described trajectory information procedure. Theabove-described program may be a program to cause a computer to performan execution time estimating procedure of estimating a period of timethat is from a time when the above-described picking apparatus receivesan instruction for starting an operation on the above-described work toa time when the operation of the above-described picking apparatus onthe above-described work is ended in the above-described executioncontrol procedure. In the above-described program, the above-describedtrajectory information generating procedure may also include a procedureof adjusting the amount of calculation based on an estimation resultobtained in the above-described execution time estimating procedure.

The above-described software or program may be a program to control thepicking apparatus. In the above-described program, the picking apparatusmay arrange, at a second position of a second container, a work pickedat a first position of a first container by using a picking hand, thesecond container being different from the above-described firstcontainer. The above-described program may be a program to cause acomputer to perform a point cloud information acquiring procedure ofacquiring point cloud information of a region including at least one ofthe above-described picking hand and the above-described work at a timebeing at least one of (i) when the above-described picking hand picksthe above-described work at the above-described first position, (ii)when the above-described picking hand moves the above-described work outof the above-described first container, (iii) when the above-describedpicking hand moves the above-described work in the above-describedsecond container, and (iv) when the above-described picking handarranges the above-described work at the above-described secondposition. The above-described program may be a program to cause acomputer to perform an abnormality detection procedure of detectingabnormality based on the point cloud information acquired in theabove-described point cloud information acquiring procedure.

The above-described software or program may be a program to control thepicking apparatus. In the above-described program, the picking apparatusmay move, by using a picking hand, one or more works from one or moresupply containers to one or more arrangement containers. Theabove-described program may be a program to cause a computer to performa trajectory information generating procedure of generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks one of the above-described one or more works storedin one supply container included in the above-described one or moresupply containers and arranges the work of the above-described one ormore works to one arrangement container included in the above-describedone or more arrangement containers. In the above-described program,after performing a process of generating trajectory informationregarding a first work included in the above-described one or more worksand before completing moving the above-described first work, theabove-described trajectory information generating procedure may includea procedure of generating trajectory information regarding a second workbeing included in the above-described one or more works and beingdifferent from the above-described first work by using an operationresult in the process of generating the trajectory information regardingthe above-described first work.

The above-described software or program may be a program to control thepicking apparatus. In the above-described program, the picking apparatusmay move, by using the picking hand, one or more works to one or moresupply containers or one or more arrangement containers. Theabove-described program may be a program to cause a computer to performa trajectory information generating procedure of generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks one of the above-described one or more works storedin one supply container included in the above-described one or moresupply containers and arranges the one or more works in one arrangementcontainer included in the above-described one or more arrangementcontainers. In the above-described program, after performing the processof generating the trajectory information regarding a first work includedin the above-described one or more works and before completing movingthe above-described first work, the above-described trajectoryinformation generating procedure may have a procedure of generatingtrajectory information regarding a second work being included in theabove-described one or more works and being different from theabove-described first work by using an operation result in the processof generating trajectory information regarding the above-described firstwork.

The above-described software or program may be a program to cause acomputer to perform a three-dimensional information acquiring procedureof acquiring three-dimensional information of a target object. Theabove-described software or program may be a program to cause a computerto perform a holding information acquiring procedure of acquiringholding information that defines a method of holding the above-describedtarget object by using the picking hand. The above-described software orprogram may be a program to cause a computer to perform an informationmanagement procedure of storing, in a storage apparatus, (i)three-dimensional information of the above-described target objectacquired by the above-described three-dimensional information acquiringunit and (ii) holding information acquired by the above-describedholding information acquiring unit in association with each other.

FIG. 2 schematically shows one example of an internal configuration ofan article management apparatus 110. In the present embodiment, thearticle management apparatus 110 includes an order information acquiringunit 210, a conveying apparatus management unit 220, a transferapparatus management unit 230, a request processing unit 240 and anarticle information storing unit 250. The order information acquiringunit 210 may be one example of an order information acquiring unit. Thearticle information storing unit 250 may be one example of the storageapparatus.

In the present embodiment, the order information acquiring unit 210acquires user order information indicating order contents of a user. Theuser order information may include identification information of anarticle that is to be a working target of the transfer apparatus 130. Asthe identification information of the article, a name of the article, acode number added to the article, a feature of an appearance of thearticle and the like can be illustrated. The user order information maybe one example of the order information.

In one embodiment, the order information acquiring unit 210 receives anorder from a user via an input apparatus (not shown in the drawings)such as a keyboard, a touch panel, a sound input apparatus and a gestureinput apparatus that are arranged in the article management apparatus110. In another embodiment, the order information acquiring unit 210receives the user order information via a communication network from acommunication terminal or another information processing apparatus of auser. The communication network may be a wired communicationtransmission path, may be a wireless communication transmission path,and may also be a combination of a wireless communication transmissionpath and a wired communication transmission path.

In the present embodiment, the conveying apparatus management unit 220manages the conveying apparatus 170. In one embodiment, the conveyingapparatus management unit 220 transfers the user order informationacquired by the order information acquiring unit 210 to the conveyingapparatus 170. In another embodiment, the conveying apparatus managementunit 220 may analyze the user order information from one or more usersto check inventory states of articles included in the user orderinformation. In still another embodiment, the conveying apparatusmanagement unit 220 may analyze the user order information from one ormore users, and if multiple types of articles are included in the userorder information, the conveying apparatus management unit 220 maydetermine a conveying sequence of the multiple types of articles to thetransfer apparatus 130 based on an inventory position of each articlesuch that the entire conveying efficiency satisfies a predeterminedcondition.

In the present embodiment, the transfer apparatus management unit 230manages the transfer apparatus 130. In one embodiment, the transferapparatus management unit 230 may analyze the user order information,and if multiple types of articles are included in the user orderinformation, the transfer apparatus management unit 230 may determine apicking sequence of the multiple types of articles to the transferapparatus 130 such that working efficiency of the transfer apparatus 130satisfies a predetermined condition. If the conveying apparatusmanagement unit 220 determines the conveying sequence of the multipletypes of articles to the transfer apparatus 130, the transfer apparatusmanagement unit 230 may determine the picking sequence of the multipletypes of articles to the transfer apparatus 130 while taking theconveying sequence into account. In another embodiment, the transferapparatus management unit 230 transfers the user order informationacquired by the order information acquiring unit 210 to the transferapparatus 130. In this case, the transfer apparatus 130 may also analyzethe user order information to determine the picking sequence of themultiple types of articles to the transfer apparatus 130.

In the present embodiment, the request processing unit 240 receivesvarious requests. The request processing unit 240 processes the receivedrequests. In one embodiment, the request processing unit 240 receives aregistration request of data regarding an article from the dataregistration apparatus 120. As the request processing unit 240 receivesthe above-described registration request, the request processing unit240 checks, for example, whether or not the data of the article to beregistered has already been stored in the article information storingunit 250. If the above-described data is not stored in the articleinformation storing unit 250, the request processing unit 240 stores theabove-described data in the article information storing unit 250. In acase where the above-described data has been stored in the articleinformation storing unit 250, if the contents of the data have beenchanged the request processing unit 240 updates the above-describeddata. On the other hand, the request processing unit 240 ends theprocessing if the contents of the data are not changed.

In another embodiment, the request processing unit 240 receives, fromthe transfer control apparatus 160, a sending request of data matchedwith a specific condition. As the request processing unit 240 receivesthe above-described sending request, the request processing unit 240accesses, for example, to the article information storing unit 250 tocheck whether the data matched with the condition is present or absent.If the data matched with the above-described condition has been storedin the article information storing unit 250, the request processing unit240 sends the data to the transfer control apparatus 160. If the datamatched with the above-described condition is not stored in the articleinformation storing unit 250, the request processing unit 240 sendsinformation indicating that no data exists to the transfer controlapparatus 160.

In the present embodiment, the article information storing unit 250stores various information regarding an article. The article informationstoring unit 250 may store the information acquired by the dataregistration apparatus 120.

FIG. 3 schematically shows one example of an internal configuration ofthe data registration apparatus 120. In the present embodiment, the dataregistration apparatus 120 includes a three-dimensional informationacquiring unit 312, a two-dimensional information acquiring unit 314, amass information acquiring unit 316 and an article informationregistration unit 320. In the present embodiment, the articleinformation registration unit 320 includes an input/output unit 330, aregistration information generating unit 340 and a registration requestsending unit 350. In the present embodiment, the registrationinformation generating unit 340 includes a coordinate information addingunit 342, a holding information adding unit 344 and a specificationinformation adding unit 346.

The article information registration unit 320 may be one example of theinformation management unit. The holding information adding unit 344 maybe one example of the holding information acquiring unit. Thespecification information adding unit 346 may be one example of thefeature information acquiring unit. The registration request sendingunit 350 may be one example of the information management unit.

In the present embodiment, the three-dimensional information acquiringunit 312 acquires three-dimensional information of a target article thatis to be a target of data registration processing. The three-dimensionalinformation may include at least one of home position information andcoordinate axis information. The three-dimensional information may alsoinclude CAD data of the target article. The three-dimensionalinformation may also include data obtained by three-dimensionallymeasuring the target article. The target article may be one example ofthe target object. The three-dimensional information acquiring unit 312may also be (i) a three-dimensional measurement apparatus in any mannerand may also be (ii) a communication interface or a storage apparatus toreceive three-dimensional information from the three-dimensionalmeasurement apparatus or another information processing apparatus.

In the present embodiment, the two-dimensional information acquiringunit 314 acquires two-dimensional information of a target article thatis to be a target of the data registration processing. Thetwo-dimensional information may include CAD data of the target article.The two-dimensional information may include image data of three or moreimages obtained by capturing the images of the target article from threeor more directions. The two-dimensional information may include imagedata of four or more images obtained by capturing the images of thetarget article from four or more directions. The two-dimensionalinformation may include image data of five or more images obtained bycapturing the images of the target article from five or more directions.The two-dimensional information may include image data of six or moreimages obtained by capturing the images of the target article from sixor more directions. The two-dimensional information acquiring unit 314may also be (i) an image capturing apparatus in any manner, and may alsobe (ii) a communication interface or a storage apparatus to receivetwo-dimensional information from the image capturing apparatus oranother information processing apparatus.

In the present embodiment, the mass information acquiring unit 316acquires mass information indicating the mass of the target article. Themass information may also include information of a center of gravityindicating a position of the center of gravity of the target article.The mass information acquiring unit 316 may also be (i) a massmeasurement apparatus in any manner, and may also be (ii) acommunication interface or a storage apparatus to receive massinformation from the mass measurement apparatus or another informationprocessing apparatus.

In the present embodiment, the article information registration unit 320stores, in the article information storing unit 250, (i) thethree-dimensional information of the target article acquired by thethree-dimensional information acquiring unit 312 and (ii) variousinformation regarding the target article in association with each other.For example, the article information registration unit 320 sends, to thearticle management apparatus 110, a request for storing (i) thethree-dimensional information of the target article acquired by thethree-dimensional information acquiring unit 312 and (ii) variousinformation regarding the target article in the article informationstoring unit 250.

In the present embodiment, the input/output unit 330 receives an inputfrom a user. Also, the input/output unit 330 outputs information to theuser. The input/output unit 330 includes an input apparatus such as akeyboard, a pointing device, a touch panel, a camera, a sound inputapparatus, a gesture input apparatus, various sensors and a GPSreceiver, for example. The input/output unit 330 may also be acommunication interface with an external storage apparatus. Theinput/output unit 330 may also be a communication interface for sendingand receiving information to/from another information processingapparatus via a communication line. Accordingly, information regarding atarget article can be acquired by accessing to a website in which amanufacturer or seller of the target article provides a service, forexample. The input/output unit 330 includes an output apparatus such asa display apparatus, a sound output apparatus and a vibration apparatus,for example. Accordingly, an input screen for the user to input theinformation regarding the target article can be presented to the user,for example.

In the present embodiment, the registration information generating unit340 generates information that is to be registered in the articlemanagement apparatus 110. For example, the registration informationgenerating unit 340 adds various information to at least one of thethree-dimensional information of the target article acquired by thethree-dimensional information acquiring unit 312, the two-dimensionalinformation of the target article acquired by the two-dimensionalinformation acquiring unit 314 and the mass information of the targetarticle acquired by the mass information acquiring unit 316, to generatethe information to be registered in the article management apparatus110.

In the present embodiment, the coordinate information adding unit 342adds at least one of the home position information and the coordinateaxis information to at least one of the three-dimensional information ofthe target article acquired by the three-dimensional informationacquiring unit 312 and the two-dimensional information of the targetarticle acquired by the two-dimensional information acquiring unit 314.The coordinate information adding unit 342 may also acquire at least oneof the home position information and the coordinate axis informationfrom the input/output unit 330.

In the present embodiment, the holding information adding unit 344 addsholding information defining a method of holding the target article byusing the picking hand to at least one of the three-dimensionalinformation of the target article acquired by the three-dimensionalinformation acquiring unit 312 and the two-dimensional information ofthe target article acquired by the two-dimensional information acquiringunit 314.

In one embodiment, the holding information adding unit 344 determines amethod of holding the target article by acquiring the holdinginformation from the input/output unit 330. In another embodiment, theholding information adding unit 344 determines the method of holding thetarget article based on (i) a shape of the target article and (ii)information associating a typical shape of the article with the methodof holding the article.

In the present embodiment, the specification information adding unit 346adds specification information indicating a specification of the targetarticle to at least one of the three-dimensional information of thetarget article acquired by the three-dimensional information acquiringunit 312 and the two-dimensional information of the target articleacquired by the two-dimensional information acquiring unit 314. As thespecification of the target article, (i) a name of the article, (ii) anidentification code added to the article, (iii) a dimension of thearticle, (iv) the mass of the article, (v) matters regarding storage,transportation or usage of the article and the like can be illustrated.

In one embodiment, the specification information adding unit 346 mayacquire the specification information from the input/output unit 330. Inanother embodiment, the specification information adding unit 346 mayacquire, based on at least one of the three-dimensional informationacquired by the three-dimensional information acquiring unit 312 and thetwo-dimensional information acquired by the two-dimensional informationacquiring unit 314, feature information for recognizing the targetarticle and include the feature information in the specificationinformation. The specification information adding unit 346 may use amethod used in an image recognition field such as face recognition andliving body recognition to acquire the feature information.

The feature information may be information regarding a feature of atleast parts of the appearance of the target article. As the feature ofthe appearance of the target article, (i) a feature regarding at leastone of a shape, a pattern and a color of the target article, (ii) a logomark added to a target article, a name of the article or anidentification code to identify the article and the like can beillustrated.

Accordingly, the registration information generating unit 340 cangenerate the information that is to be registered in the articlemanagement apparatus 110. Note that the information that is to beregistered in the article management apparatus 110 may at least includeat least one of the three-dimensional information of the target articleacquired by the three-dimensional information acquiring unit 312 and thetwo-dimensional information of the target article acquired by thetwo-dimensional information acquiring unit 314, or may not include themass information, the coordinate information, the holding informationand the specification information.

In one embodiment, the information to be registered in the articlemanagement apparatus 110 includes the three-dimensional information andthe holding information of the target article. The information to beregistered in the article management apparatus 110 may also include (i)the three-dimensional information and the holding information of thetarget article and (ii) at least one of the feature information and theidentification information of the target article. The information to beregistered in the article management apparatus 110 may also include (i)the three-dimensional information and the holding information of thetarget article and (ii) at least one of the mass information, thecoordinate information and the specification information of the targetarticle. The information to be registered in the article managementapparatus 110 may also include (i) the three-dimensional information andthe holding information of the target article, (ii) at least one of thefeature information and the identification information of the targetarticle, and (iii) at least one of the mass information, the coordinateinformation and the specification information of the target article.

In another embodiment, the information to be registered in the articlemanagement apparatus 110 includes the three-dimensional information, thetwo-dimensional information and the holding information of the targetarticle. The information to be registered in the article managementapparatus 110 may also include (i) the three-dimensional information,the two-dimensional information and the holding information of thetarget article, and (ii) at least one of the feature information and theidentification information of the target article. The information thatis to be registered in the article management apparatus 110 may alsoinclude (i) the three-dimensional information, the two-dimensionalinformation and the holding information of the target article, and (ii)at least one of the mass information, the coordinate information and thespecification information of the target article. The information that isto be registered in the article management apparatus 110 may alsoinclude (i) the three-dimensional information, the two-dimensionalinformation and the holding information of the target article, (ii) atleast one of the feature information and the identification informationof the target article, and (iii) at least one of the mass information,the coordinate information and the specification information of thetarget article.

In the present embodiment, the registration request sending unit 350stores the registration information generated by the registrationinformation generating unit 340 in the article management apparatus 110.For example, the registration request sending unit 350 sends, to thearticle management apparatus 110, a registration request for storing theregistration information generated by the registration informationgenerating unit 340 in the article information storing unit 250.

In the present embodiment, a case in which the article informationstoring unit 250 that is where the registration information is to beregistered is arranged in the article management apparatus 110 isdescribed. However, the article information storing unit 250 is notlimited to the present embodiment. In another embodiment, the articleinformation storing unit 250 may also be arranged in the dataregistration apparatus 120.

FIG. 4 schematically shows one example of an internal configuration of arobot 140. In the present embodiment, the robot 140 includes amanipulator 410, a driving unit 420 and a drive control unit 430. In thepresent embodiment, the manipulator 410 includes a picking hand 412 anda robot arm 414.

In the present embodiment, the picking hand 412 holds the work 10. Theholding manner of the picking hand 412 is not particularly limited. Asthe holding manner of the picking hand 412, a manner of attracting thework 10 with a low pressure, a manner of holding the work 10 by aplurality of finger modules and the like can be illustrated. The pickinghand 412 may also be exchanged in accordance with the working contents.The picking hand 412 may have various sensors such as a contact sensor,a mass measurement sensor and a load measurement sensor.

In the present embodiment, the robot arm 414 is coupled with the pickinghand 412 to adjust a position and a posture of the picking hand 412.Although the structure of the robot arm 414 is not particularly limited,a multijoint robot arm having a plurality of joints is used, forexample. The robot arm 414 may have various sensors such as a massmeasurement sensor and a load measurement sensor. The load measurementsensor may be a sensor to measure torque, a current value or a voltagevalue for the driving unit 420 that drives each joint of the robot arm414.

In the present embodiment, the driving unit 420 drives the manipulator410. In one embodiment, the driving unit 420 drives the picking hand412. For example, the driving unit 420 supplies air to the picking hand412, or sucks air from the picking hand 412. In another embodiment, thedriving unit 420 drives the robot arm 414. For example, the driving unit420 adjusts an angle of each joint of the robot arm 414. The drivingunit 420 may be a motor. The driving unit 420 may have a plurality ofmotors.

In the present embodiment, the drive control unit 430 controls thedriving unit 420. For example, the drive control unit 430 receives, fromthe transfer control apparatus 160, an instruction for controlling anoperation of the robot 140. The drive control unit 430 controls thedriving unit 420 based on the instruction from the transfer controlapparatus 160. The instruction for controlling the operation of therobot 140 may be information indicating a trajectory of the picking hand412.

As the information indicating the trajectory of the picking hand 412,(i) information indicating an initial arrangement, a passingarrangement, a final arrangement and move time of the picking hand 412,(ii) information indicating an angle of each joint and move time of therobot arm 414 in the initial arrangement, the passing arrangement andthe final arrangement, (iii) information indicating an arrangement ofthe picking hand 412 at each time during the move time, (iv) informationindicating an angle of each joint of the robot arm 414 at each timeduring the move time, and the like can be illustrated. The arrangementof the picking hand 412 may be identified by the position and theposture of the picking hand 412, for example.

FIG. 5 schematically shows one example of an internal configuration ofthe transfer control apparatus 160. In the present embodiment, thetransfer control apparatus 160 includes a recognition informationstoring unit 510, a recognition unit 520, a planning unit 530, anexecution unit 540 and a concurrent processing control unit 550.

The recognition information storing unit 510 may be one example of theholding information storing unit. The recognition unit 520 may be oneexample of the order information acquiring unit, the image acquiringunit, the point cloud information acquiring unit, the holdinginformation extracting unit and the container information acquiringunit. The planning unit 530 may be one example of the control device andthe trajectory information generating unit. The execution unit 540 maybe one example of the control device and the execution control unit. Theconcurrent processing control unit 550 may be one example of theexecution time estimating unit and the constraint condition settingunit.

In the present embodiment, the recognition information storing unit 510stores various information regarding the recognition processing in therecognition unit 520. In one embodiment, the recognition informationstoring unit 510 stores information that is used for the recognitionprocessing in the recognition unit 520. For example, the recognitioninformation storing unit 510 stores the identification information toidentify each of one or more types of articles and at least one of thethree-dimensional information, the two-dimensional information and thefeature information of each article in association with each other.Accordingly, the recognition unit 520 can recognize articles stored inthe supply pallet 12 and recognize a work 10 that is to be a workingtarget of the transfer apparatus 130 from among the identified articles.

The recognition information storing unit 510 may further store at leastone of the home position information, the coordinate axis information,the holding information and the specification information of eacharticle. These pieces of information are used by the planning unit 530,for example. For example, the recognition information storing unit 510stores the identification information to identify each of one or moretypes of articles, at least one of the three-dimensional information,the two-dimensional information and the feature information of eacharticle, and at least one of the home position information, thecoordinate axis information, the holding information and thespecification information of each article in association with eachother.

In another embodiment, the recognition information storing unit 510stores the information obtained by the recognition processing in therecognition unit 520. For example, the recognition information storingunit 510 stores (i) a recognition result of each of one or more supplypallets 12 arranged at working positions of the transfer apparatus 130,(ii) a recognition result of an article stored in each of the one ormore supply pallets 12, (iii) a recognition result of a work 10 that isto be the working target of the transfer apparatus 130 among therecognized articles, (iv) a point cloud of surfaces of the one or moresupply pallets 12, (v) a recognition result of each of one or moredelivery pallets 14 arranged at working positions of the transferapparatus 130, (vi) a recognition result of an article stored in each ofthe one or more delivery pallets 14, (vii) a recognition result of thebeing-transferred work 10 on each of the one or more delivery pallets14, (viii) a point cloud of the surfaces of the one or more deliverypallets 14, and the like. The above-described point cloud may be athree-dimensional point cloud.

The recognition information storing unit 510 may store informationrequired for the working of the transfer apparatus 130 among theinformation regarding the articles being stored in the articleinformation storing unit 250. For example, the recognition informationstoring unit 510 stores information regarding the article that waspreviously the working target of the transfer apparatus 130, informationregarding the article that is often the working target of the transferapparatus 130, information regarding an article that is scheduled to bethe working target of the transfer apparatus 130, information regardingan article that is estimated to be the working target of the transferapparatus 130 in a near future according to a previous working history,and the like. Accordingly, the recognition unit 520 can extract targetinformation at high speed.

The recognition information storing unit 510 may be realized by a mediumbeing different from the article information storing unit 250 that iswhere the data acquired by the data registration apparatus 120 is to beregistered. For example, the recognition information storing unit 510 isrealized by a medium to which the recognition unit 520 can access athigher speed than the article information storing unit 250. Accordingly,the recognition unit 520 can extract the target information at highspeed.

In the present embodiment, the recognition unit 520 performs therecognition processing. For example, the planning unit 530 acquires theworking information indicating the working contents of the transferapparatus 130 from the article management apparatus 110. The workinginformation includes the information indicating the picking sequence ofthe one or more articles to the transfer apparatus 130, for example. Theworking information may also be the user order information. The workinginformation may be one example of the order information. Also, therecognition unit 520 acquires, from the image capturing apparatus 152and the image capturing apparatus 154, image data of each of the one ormore supply pallets 12 and the one or more delivery pallets 14 arrangedat the working positions of the transfer apparatus 130.

In the present embodiment, the recognition unit 520 recognize, withreference to the information stored in the recognition informationstoring unit 510 or the article information storing unit 250, the supplypallets 12, the articles stored in the supply pallets 12, the deliverypallets 14, the articles stored in the delivery pallets 14 and the work10 being the working target of the transfer apparatus 130. Therecognition unit 520 may send the information indicating the recognitionresult to the planning unit 530. The recognition unit 520 may send theinformation indicating the recognition result to the execution unit 540.The recognition unit 520 may store the information indicating therecognition result in the recognition information storing unit 510.Details of the recognition unit 520 are described below.

In the present embodiment, the planning unit 530 performs the planningprocessing. For example, the planning unit 530 creates a working plan ofthe transfer apparatus 130. More specifically, the planning unit 530generates trajectory information prescribing a trajectory on which thepicking hand 412 picks the work 10 stored in a specific supply pallet 12and arranges the work on a specific delivery pallet 14. The specificsupply pallet 12 may be one example of one supply container included inone or more supply containers. The specific delivery pallet 14 may beone example of one arrangement container included in one or morearrangement containers.

The trajectory information prescribes a trajectory on which the pickinghand 412 picks the work 10 at a first position and arranges the work ata second position, for example. The second position may be a positionbeing different from the first position. The first position may be anyposition or a specific position of the supply pallet 12. The secondposition may be any position or a specific position of the deliverypallet 14.

In the present embodiment, the planning unit 530 generates thetrajectory information based on the working information from the articlemanagement apparatus 110 and the recognition result of the recognitionunit 520 with reference to the holding information stored in therecognition information storing unit 510 or the article informationstoring unit 250. Accordingly, the planning unit 530 can create aworking plan of the transfer apparatus 130. The planning unit 530 maygenerate control information for controlling at least one of the pickinghand 412 and the robot arm 414 based on the generated trajectoryinformation. The planning unit 530 may send the generated controlinformation to the execution unit 540. Details of the planning unit 530are described below.

In the present embodiment, the execution unit 540 performs the executionprocessing. For example, the execution unit 540 receives the controlinformation from the planning unit 530. The execution unit 540 sends thecontrol information to the drive control unit 430 of the robot 140 tooperate the robot 140 based on the trajectory information generated bythe planning unit 530. Also, the execution unit 540 monitors theoperation of the robot 140, the supply pallet 12 and the delivery pallet14. The control information may be one example of an instruction to therobot 140. The control information may be one example of an instructionfor causing the robot 140 to start an operation on the work 10. Detailsof the execution unit 540 are described below.

In the present embodiment, the recognition unit 520 may performconcurrently or in parallel parts of a plurality of steps configuringthe recognition processing. Similarly, the planning unit 530 may performconcurrently or in parallel parts of a plurality of steps configuringthe planning processing. The execution unit 540 may perform concurrentlyor in parallel parts of a plurality of steps configuring the executionprocessing. Also, the transfer control apparatus 160 may performconcurrently or in parallel the recognition processing, the planningprocessing and the execution processing regarding a plurality of works10. For example, the transfer control apparatus 160 performs planningprocessing of a second work and recognition processing of a third workwhile performing execution processing of a first work. Theabove-described concurrent processing or parallel processing can berealized by a pipeline processing technology, for example.

In the present embodiment, the concurrent processing or concurrentprocessing control unit 550 controls the parallel processing in thetransfer control apparatus 160. In one embodiment, the concurrentprocessing control unit 550 controls the concurrent processing orparallel processing in each of the recognition unit 520, the planningunit 530 and the execution unit 540. In another embodiment, in a case inwhich the transfer control apparatus 160 performs concurrently or inparallel the processing regarding the plurality of works 10, theconcurrent processing control unit 550 controls the recognition unit520, the planning unit 530 and the execution unit 540 such that theinformation can be smoothly exchanged between the recognition unit 520,the planning unit 530 and the execution unit 540.

First Embodiment of Controlling Other Elements by Concurrent ProcessingControl Unit 550

In one embodiment, the concurrent processing control unit 550 estimatesa period of time that is from a time when the robot 140 receives thecontrol information for starting the operation on the work 10 to a timewhen the operation of the robot 140 on the work 10 is ended. Forexample, the trajectory information includes information regarding (i)an initial position and an initial posture of the picking hand 412, (ii)a position and a posture of the picking hand 412 picking the work 10 onthe supply pallet 12, (iii) a position and a posture of the picking hand412 arranging the work 10 on the delivery pallet 14, and (iv) move timeof the work 10. Here, the concurrent processing control unit 550 mayestimate the above-described time based on the trajectory informationgenerated by the planning unit 530.

The concurrent processing control unit 550 may determine a constraintcondition regarding an amount or duration time of calculation of theprocessing in at least one of the recognition unit 520, the planningunit 530 and the execution unit 540 based on the above-describedestimation result. For example, the concurrent processing control unit550 determines the constraint condition regarding the amount or durationtime of calculation of the processing in at least one of a plurality ofsteps configuring the recognition processing in the recognition unit 520based on the above-described estimation result. The recognition unit 520may adjust the amount or duration time of calculation based on theabove-described constraint condition. Similarly, the concurrentprocessing control unit 550 may determine the constraint conditionregarding the amount or duration time of calculation of the processingin at least one of a plurality of steps configuring the planningprocessing in the planning unit 530 based on the above-describedestimation result. The planning unit 530 may adjust the amount orduration time of calculation based on the above-described constraintcondition.

The constraint condition may be at least one of (i) the maximum value ofan amount or duration time of calculation that is until progress of eachprocessing reaches a preset milestone and (ii) the maximum value of anamount or duration time of calculation that is until each processing iscompleted. The above-described maximum value may be determined based onthe above-described estimation result.

For example, in a case in which the transfer control apparatus 160performs the planning processing of the second work and the recognitionprocessing of the third work while performing the execution processingof the first work, the concurrent processing control unit 550 determinesthe maximum value of the amount or duration time of calculation of theplanning processing of the second work such that the planning processingof the second work in the planning unit 530 is ended before theabove-described estimate time is expired or elapsed. Accordingly, thetrajectory information or control information regarding the second workcan be generated before the execution processing of the first work inthe execution unit 540 is ended.

Similarly, the concurrent processing control unit 550 may determine themaximum value of the amount or duration time of calculation of therecognition processing of the third work such that the recognitionprocessing of the third work in the recognition unit 520 is ended beforethe planning processing of the second work in the planning unit 530 isended. The concurrent processing control unit 550 may also determinesthe maximum value of the amount or duration time of calculation of therecognition processing of the third work such that the recognitionprocessing of the third work in the recognition unit 520 ends before theexecution processing of the first work in the execution unit 540 isended.

Second Embodiment of Controlling Other Elements by Concurrent ProcessingControl Unit 550

In another embodiment, the concurrent processing control unit 550adjusts the time between the processing in the recognition unit 520, theprocessing in the planning unit 530 and the processing in the executionunit 540. For example, the concurrent processing control unit 550adjusts the time between the recognition processing in the recognitionunit 520 and the execution processing in the execution unit 540. Morespecifically, the concurrent processing control unit 550 adjusts thetime between a step in which the recognition unit 520 acquires the pointcloud information and a step in which the execution unit 540 detects theabnormality.

For example, in a case in which the transfer control apparatus 160performs the planning processing of the second work and the recognitionprocessing of the third work while performing the execution processingof the first work, the execution unit 540 acquires the point cloudinformation of the supply pallet 12 in periods of time that are beforeand after the picking hand 412 picks the first work, to determinewhether the abnormality is present or absent. Also, the execution unit540 acquires the point cloud information of the delivery pallet 14 inperiods of time that are before and after the picking hand 412 arrangesthe first work, to determine whether the abnormality is present orabsent. Accordingly, occurrence of the abnormality can be rapidlydetected. On the other hand, in the recognition processing of therecognition unit 520, for example, the process of recognizing the thirdwork and the process of acquiring the point cloud information of thesupply pallet 12 or the delivery pallet 14 are performed concurrently.Because the duration time of calculation for the process of acquiringthe point cloud information is shorter compared to the process ofrecognizing the work, the recognition unit 520 can perform the processof acquiring the point cloud information for multiple times whileperforming the process of recognizing the work.

Here, the concurrent processing control unit 550 controls therecognition unit 520, for example, to adjust the time at which therecognition unit 520 starts the step of acquiring the point cloudinformation of the supply pallet 12 or the delivery pallet 14 inaccordance with the progress of the execution processing of the firstwork by the execution unit 540. According to one embodiment, theconcurrent processing control unit 550 estimates the time at which thepicking hand 412 picks the first work or estimates the picking hand 412,to control the recognition unit 520 such that the recognition unit 520starts the step of acquiring the point cloud information in accordancewith the time. The concurrent processing control unit 550 may estimatethe above-described time based on the trajectory information generatedby the planning unit 530. According to another embodiment, at the timewhen the picking hand 412 picks the first work or the time when thepicking hand 412 arranges the first work, if the step of acquiring thepoint cloud information is being performed, the recognition unit 520suspends the step and newly starts the process of acquiring the pointcloud information.

FIG. 6 schematically shows one example of an internal configuration ofthe recognition unit 520. In the present embodiment, the recognitionunit 520 includes a position and posture information generating unit 610and a point cloud information generating unit 620. The position andposture information generating unit 610 may be one example of an imageacquiring unit, a point cloud information acquiring unit, a containerinformation acquiring unit and a holding information extracting unit.The point cloud information generating unit 620 may be one example ofthe image acquiring unit and the point cloud information acquiring unit.

In the present embodiment, the position and posture informationgenerating unit 610 acquires, from the image capturing apparatus 152 andthe image capturing apparatus 154, image data of each of the one or moresupply pallets 12 and the one or more delivery pallets 14 arranged atthe working positions of the transfer apparatus 130. The position andposture information generating unit 610 may also acquire, from the imagecapturing apparatus 152 and the image capturing apparatus 154, pointcloud data of each of the one or more supply pallets 12 and the one ormore delivery pallets 14 arranged at the working positions of thetransfer apparatus 130. The above-described point cloud data may bethree-dimensional point cloud data.

In the present embodiment, the position and posture informationgenerating unit 610 acquires working information from the articlemanagement apparatus 110. As described above, the working informationincludes information indicating the picking sequence of the one or morearticles by the transfer apparatus 130, for example. The workinginformation may include identification information of an article that isto be the working target of the transfer apparatus 130. The workinginformation, information indicating an article that is the workingtarget, information indicating a supply pallet 12 that is the movesource and information indicating a delivery pallet 14 that is the movedestination may also be stored in association with each other. In a casein which the supply pallet 12 stores a single type of article, theidentification information of the supply pallet 12 may also be used asthe identification information of the article that is the working targetof the transfer apparatus 130.

In the present embodiment, the position and posture informationgenerating unit 610 recognizes at least one (which may be referred to asan element) of (i) the supply pallet 12, (ii) at least some of the oneor more articles stored in the supply pallet 12, (iii) the deliverypallet 14 and (iv) at least some of the one or more articles stored inthe delivery pallet 14. The position and posture information generatingunit 610 may also recognize the work 10 that is the working target ofthe transfer apparatus 130. The position and posture informationgenerating unit 610 recognizes a position and a posture of eachrecognized element. The position and posture information generating unit610 generates the position and posture information indicating theinformation of the position and the posture of each recognized element.The position and posture information generating unit 610 sends thegenerated position and posture information to the planning unit 530.

The position and posture information generating unit 610 recognizes theposition and the posture of each element according the followingprocedures, for example. For example, the position and postureinformation generating unit 610 first analyzes the working informationto identify the article being the working target. Next, the position andposture information generating unit 610 accesses to the recognitioninformation storing unit 510 and extracts, by taking the identificationinformation of the article being the working target as a key, from amongthe information stored in the recognition information storing unit 510,at least one of the two-dimensional information, the three-dimensionalinformation and the feature information of the article associated withthe identification information. At this time, the position and postureinformation generating unit 610 may also extract holding informationthat is associated with the identification information of the articlebeing the working target.

The recognition information storing unit 510 may not store informationof some articles of the articles included in the working information.Here, in a case in which the information regarding the article being theworking target is not stored in the recognition information storing unit510, the position and posture information generating unit 610 accessesto the article information storing unit 250 and extracts, by taking theidentification information of the article as the key, from among theinformation stored in the article information storing unit 250, at leastone of the two-dimensional information, the three-dimensionalinformation and the feature information of the article associated withthe identification information. At this time, the position and postureinformation generating unit 610 may also extract holding informationassociated with the identification information of the article being theworking target. When the position and posture information generatingunit 610 acquires information from the article information storing unit250, the position and posture information generating unit 610 may store,in the recognition information storing unit 510, the information and theidentification information of the above-described article and store theinformation in association with each other.

Next, the position and posture information generating unit 610 acquires,from the image capturing apparatus 152 and the image capturing apparatus154, image data of each of the one or more supply pallets 12 and the oneor more delivery pallets 14 arranged at the working positions of thetransfer apparatus 130. The recognition unit 520 may also acquire, fromthe image capturing apparatus 152 and the image capturing apparatus 154,point cloud data of each of the one or more supply pallets 12 and theone or more delivery pallets 14 arranged at the working positions of thetransfer apparatus 130. Accordingly, the position and postureinformation generating unit 610 can acquire the information regardingthe state of the surface at least one of the supply pallet 12 and thedelivery pallet 14.

Next, the position and posture information generating unit 610 analyzesthe image data or the point cloud data of each of the one or more supplypallets 12 by using at least one of the two-dimensional information, thethree-dimensional information and the feature information of the articlebeing the working target to recognize (i) the position and the postureof each supply pallet 12 and (ii) the positions and the postures of atleast some of the one or more articles stored in each supply pallet 12.Similarly, the position and posture information generating unit 610recognizes (i) the position and the posture of the delivery pallet 14and (ii) the positions and the postures of at least some of the one ormore articles stored in the delivery pallet 14.

The position and posture information generating unit 610 may also selectthe work 10 being the working target of the transfer apparatus 130 fromamong the articles stored in the one or more supply pallets 12. Theposition and posture information generating unit 610 may select the work10 being the working target of the transfer apparatus 130 from among thearticles whose positions and postures having already been recognized.Note that the process of selecting the work 10 may also be performed inthe planning unit 530.

In the present embodiment, the point cloud information generating unit620 generates the point cloud information of at least one of the supplypallet 12 and the delivery pallet 14. The point cloud informationgenerating unit 620 generates the point cloud information according tothe following procedures, for example. In one embodiment, the pointcloud information generating unit 620 first acquires, from the imagecapturing apparatus 152 and the image capturing apparatus 154, pointcloud data of each of the one or more supply pallets 12 and the one ormore delivery pallets 14 arranged at the working positions of thetransfer apparatus 130. Next, the point cloud information generatingunit 620 associates the point cloud data of each of the one or moresupply pallets 12 and the one or more delivery pallets 14 with theidentification information thereof to generate the point cloudinformation of each of the one or more supply pallets 12 and the one ormore delivery pallets 14.

In another embodiment, the point cloud information generating unit 620first acquires, from the image capturing apparatus 152 and the imagecapturing apparatus 154, image data of each of the one or more supplypallets 12 and the one or more delivery pallets 14 arranged at theworking positions of the transfer apparatus 130. Next, the point cloudinformation generating unit 620 acquires the point cloud data of each ofthe one or more supply pallets 12 and the one or more delivery pallets14 based on the image data of each of the one or more supply pallets 12and the one or more delivery pallets 14 arranged at the workingpositions of the transfer apparatus 130. Next, the point cloudinformation generating unit 620 associates the point cloud data of eachof the one or more supply pallets 12 and the one or more deliverypallets 14 with the identification information thereof to generate thepoint cloud information of each of the one or more supply pallets 12 andthe one or more delivery pallets 14.

In the present embodiment, the point cloud information generating unit620 sends the point cloud information to the planning unit 530. Thepoint cloud information generating unit 620 may also send the pointcloud information to the execution unit 540. The point cloud informationgenerating unit 620 may also generate the point cloud information inaccordance with an instruction or a request from the execution unit 540or the concurrent processing control unit 550, and send the point cloudinformation to the execution unit 540.

FIG. 7 schematically shows one example of an internal configuration ofthe planning unit 530. In the present embodiment, the planning unit 530includes a holding planning unit 710, an arrangement planning unit 720,an evaluation information storing unit 730, a trajectory candidateextracting unit 740, a trajectory planning unit 750 and a controlinformation generating unit 760.

The holding planning unit 710 may be one example of a first candidateinformation generating unit. The arrangement planning unit 720 may beone example of a second candidate information generating unit. Thetrajectory candidate extracting unit 740 may be one example of the firstcandidate information acquiring unit, the second candidate informationacquiring unit and the extracting unit. The trajectory planning unit 750may be one example of the trajectory information generating unit, thetrajectory calculating unit, the determining unit, the trajectorydetermining unit, and the interpolation trajectory generating unit.

In the present embodiment, the holding planning unit 710 plans a holdingmethod of the work 10 by the picking hand 412. Specifically, the holdingplanning unit 710 determines the position and the posture of the pickinghand 412 when the picking hand 412 holds the work 10 stored in thesupply pallet 12. For example, the holding planning unit 710 extractsone or more postures (which may be referred to as holding candidates)that the picking hand 412 can take on a position (which may be referredto as a picking position) on which the picking hand 412 holds the work10, with reference to (i) the position and posture information of thesupply pallet 12, (ii) the position and posture information of the oneor more articles stored in the supply pallet 12, (iii) the point cloudinformation of the supply pallet 12 and (iv) the holding informationacquired by the recognition unit 520 from the recognition informationstoring unit 510 or the article management apparatus 110. The pickingposition may be one example of a first position.

For example, if the picking hand 412 holds, according to the holdingmethod indicated by the holding information, an article that is of atleast some of the one or more articles stored in the supply pallet 12,the holding planning unit 710 determines whether or not the picking hand412 interferes with the supply pallet 12 and other articles. When theholding planning unit 710 determines that the picking hand 412 does notinterfere with the supply pallet 12 and other articles, the holdingplanning unit 710 extracts the position and the posture of the pickinghand 412 at that time as the holding candidate. The holding planningunit 710 repeats the above-described determining processing within arange of a constraint condition regarding the amount or duration time ofcalculation set by the concurrent processing control unit 550.Accordingly, the holding planning unit 710 can generate the informationregarding one or more holding candidates. The holding planning unit 710sends the information regarding the one or more holding candidates tothe trajectory candidate extracting unit 740. The information regardingthe one or more holding candidates may be one example of the firstcandidate information.

In the present embodiment, the arrangement planning unit 720 plans anarrangement method of the work 10 by the picking hand 412. Specifically,the arrangement planning unit 720 determines the position and theposture of the picking hand 412 when the picking hand 412 arranges thework 10 on the delivery pallet 14. For example, the arrangement planningunit 720 extracts one or more postures (which may be referred to asarrangement candidates) that the picking hand 412 can take on a position(which may be referred to as an arrangement position) on which thepicking hand 412 arranges the work 10, with reference to (i) theposition and posture information of the delivery pallet 14, (ii) theposition and posture information of the one or more articles stored inthe delivery pallet 14, (iii) the point cloud information of thedelivery pallet 14 and (iv) the holding information acquired by therecognition unit 520 from the recognition information storing unit 510or the article management apparatus 110. The arrangement position may beone example of a second position.

For example, if the picking hand 412 arranges, according to the holdingmethod indicated by the holding information, the work 10 at any positionor a specific position of the delivery pallet 14, the arrangementplanning unit 720 determines whether or not the picking hand 412 and thework 10 interfere with the delivery pallet 14 and other articles. Whenthe arrangement planning unit 720 determines that the picking hand 412and the work 10 do not interfere with the delivery pallet 14 and theother articles, the arrangement planning unit 720 extracts the positionand the posture of the picking hand 412 at that time as the arrangementcandidate. The arrangement planning unit 720 repeats the above-describeddetermining processing within a range of the constraint conditionregarding the amount or duration time of calculation set by theconcurrent processing control unit 550. Accordingly, the arrangementplanning unit 720 can generate the information regarding one or morearrangement candidates. Note that the arrangement planning unit 720 mayalso repeat the above-described determining processing until allarticles included in the user order information are arranged on thedelivery pallets 14. The arrangement planning unit 720 sends theinformation regarding the one or more arrangement candidates to thetrajectory candidate extracting unit 740. The information regarding theone or more arrangement candidates may be one example of the secondcandidate information.

In the present embodiment, the evaluation information storing unit 730stores information (which may be referred to as evaluation information)regarding a determination reference for extracting a specific trajectorycandidate from among one or more trajectory candidates during a processof extracting the trajectory candidates by the trajectory candidateextracting unit 740. As the evaluation information, (i) information forevaluating a period of time during which the picking hand 412 moves fromone position to another position, (ii) information for evaluating aperiod of time during which the picking hand 412 moves from one positionto another position with a posture, (iii) information for evaluating aperiod of time during which the picking hand 412 moves from one positionwith one posture to another position and (iv) information for evaluatinga period of time during which the picking hand 412 moves from oneposition with one posture to another position with another posture canbe illustrated. The period of time of the movement from one position toanother position may be evaluated based on an actual measurement valueduring a trial operation, a previous operation history, empirical lawsand the like, for example.

In the present embodiment, the trajectory candidate extracting unit 740extracts a specific candidate (which may be referred to as a trajectorycandidate) from among a plurality of candidates associated with thetrajectorys of the picking hand 412. The trajectory candidate isindicated according to an initial arrangement and a final arrangement ofthe picking hand 412, for example. The trajectory candidate may also beindicated according to the initial arrangement, one or more passingarrangements, and the final arrangement of the picking hand 412. Each ofthe initial arrangement, the one or more passing arrangements, and thefinal arrangement of the picking hand 412 may also include informationregarding the positions and postures of the picking hand 412. At leastone of the initial arrangement, the one or more passing arrangements,and the final arrangement of the picking hand 412 may includeinformation regarding the positions of the picking hand 412 and also,may not include the information regarding the postures of the pickinghand 412.

The trajectory candidate extracting unit 740 extracts the trajectorycandidates according to the following procedures, for example. Thetrajectory candidate extracting unit 740 first receives informationregarding one or more holding candidates from the holding planning unit710. Accordingly, for example, the trajectory candidate extracting unit740 acquires M holding candidates, where M is an integer being 0 orequal to or more than 1. Also, the trajectory candidate extracting unit740 receives information regarding the one or more arrangementcandidates from the arrangement planning unit 720. Accordingly, forexample, the trajectory candidate extracting unit 740 acquires Narrangement candidates, where N is an integer being 0 or equal to ormore than 1.

At this time, the trajectory candidate extracting unit 740 may alsoacquire information indicating a position (which may be referred to as astarting position) of the picking hand 412 when the robot 140 startsworking on a work or a position and a posture of the picking hand 412when the robot 140 starts working on the work. The starting position maybe one example of a third position. In one embodiment, the startingposition of the picking hand 412 may be an immediately precedingarrangement position of the work. Also, the posture of the picking hand412 on the starting position may be a posture of the picking hand 412 onthe immediately preceding arrangement position of the work. In anotherembodiment, the position and the posture of the picking hand 412 whenthe robot 140 starts working on the work may also be predetermined.

Next, the trajectory candidate extracting unit 740 extracts onecombination from among M×N combinations regarding M holding candidatesand N arrangement candidates. The trajectory candidate extracting unit740 sends the information indicating the extracted combination, as thetrajectory candidates, to the trajectory planning unit 750.

The trajectory candidate extracting unit 740 extracts the combinationbeing the trajectory candidate according to the following procedures,for example. First, the trajectory candidate extracting unit 740calculates, for a combination of one of the M holding candidates and oneof the N arrangement candidates, an evaluation function for a period oftime during which the picking hand 412 moves from holding position andposture indicated by the selected holding candidate to arrangementposition and posture indicated by the selected arrangement candidate.The trajectory candidate extracting unit 740 may calculate theabove-described evaluation function with reference to the informationstored in the evaluation information storing unit 730. For example, theevaluation function includes an evaluation item regarding an estimatedvalue of a period of time that is from (i) a time when the picking hand412 picks the work at a holding position with the posture indicated bythe selected holding candidate to (ii) a time when the picking hand 412arranges the work at an arrangement position with the posture indicatedby the selected arrangement candidate.

Note that the trajectory candidate extracting unit 740 may alsocalculate the above-described evaluation function for a trajectory towhich the starting position and posture are set as the initialarrangement, the holding position and posture are set as the passingarrangement and the arrangement position and posture are set as thefinal arrangement. At this time, the evaluation function may include anevaluation item regarding an estimated value of a period of time duringwhich (i) the picking hand 412 moves from the starting position to theholding position, (ii) the picking hand 412 picks a work at the holdingposition with a posture indicated by the selected holding candidate and(iii) the picking hand 412 arranges the work at the arrangement positionwith a posture indicated by the selected arrangement candidate.

Next, the trajectory candidate extracting unit 740 repeats the processof calculating the evaluation function for each of the combinations ofthe holding candidates and the arrangement candidates within a range ofthe constraint condition regarding the amount or duration time ofcalculation set by the concurrent processing control unit 550. Thetrajectory candidate extracting unit 740 selects a combination with thehighest evaluation, as the trajectory candidate, among the combinationscalculated according to the evaluation function. For example, when theevaluation function is an estimated value of a period of time duringwhich the picking hand 412 moves from the starting position to thearrangement position, if the estimated value of the combination issmaller, the evaluation for the combination is higher. Note that whenthe calculated evaluation function satisfies the predeterminedcondition, even if the above-described constraint condition issatisfied, the trajectory candidate extracting unit 740 may also cancelthe process of calculating the evaluation function and select acombination of the holding candidate and the arrangement candidate atthat time, as the trajectory candidate.

In the present embodiment, the trajectory planning unit 750 generatesthe trajectory information prescribing the trajectory on which thepicking hand 412 picks a work at a holding position and arranges thework at an arrangement position. The trajectory planning unit 750 mayalso generate trajectory information prescribing a trajectory on whichthe picking hand 412 moves from the starting position to a holdingposition, picks a work at the holding position, and arranges the work toan arrangement position. The trajectory planning unit 750 sends thegenerated trajectory information to the control information generatingunit 760.

The trajectory planning unit 750 generates the trajectory informationaccording to the following procedures, for example. The trajectoryplanning unit 750 first receives information indicating a combination ofa holding candidate and an arrangement candidate from the trajectorycandidate extracting unit 740. Next, the trajectory planning unit 750calculates a trajectory on which (i) the picking hand 412 picks, at theholding position, a work with a posture indicated by the holdingcandidate included in the combination extracted by the trajectorycandidate extracting unit 740 and (ii) the picking hand 412 arranges, onthe arrangement position, the work with a posture indicated by thearrangement candidate included in the combination extracted by thetrajectory candidate extracting unit 740. At this time, the trajectoryplanning unit 750 may also calculate a trajectory on which (i) thepicking hand 412 moves from the starting position to the holdingposition, (ii) the picking hand 412 picks, at the holding position, awork with a posture indicated by the holding candidate included in thecombination extracted by the trajectory candidate extracting unit 740and (iii) the picking hand 412 arranges, at the arrangement position,the work with a posture indicated by the arrangement candidate includedin the combination extracted by the trajectory candidate extracting unit740.

The trajectory planning unit 750 may calculate the above-describedtrajectory from the combination of the holding candidate and thearrangement candidate according to a predetermined algorithm. Thetrajectory planning unit 750 may also calculate the above-describedtrajectory from the combination of the holding candidate and thearrangement candidate such that the generated trajectory satisfies apredetermined condition.

Next, the trajectory planning unit 750 determines whether theinterference is present or absent on the calculated trajectory. Forexample, the trajectory planning unit 750 determines whether or notanother object (for example, an obstacle, another device, anotherelement of the robot 140, and the like) interferes with the picking hand412 when the picking hand 412 moves along the calculated trajectory.Note that during a step after the picking hand 412 picks the work 10,the trajectory planning unit 750 determines whether or not anotherobject interferes with the picking hand 412 and the work 10.

The trajectory planning unit 750 may also divide the calculatedtrajectory into a plurality of zones and determine whether theinterference is present or absent for each zone. The process ofdetermining the interference for each zone may be performed concurrentlyor in parallel. When there is even only one zone in which theinterference occurs, the trajectory planning unit 750 can determine thatthe interference occurs on the trajectory even if the determiningprocessing for another zone is not ended. Accordingly, the determinationtime can be shortened.

The trajectory planning unit 750 determines the trajectory of thepicking hand 412 based on the above-described determination result. Forexample, the trajectory planning unit 750 generates an interpolationtrajectory of the trajectory if the trajectory planning unit 750determines that no interference occurs on the calculated trajectory.Accordingly, an excessive acceleration can be prevented from beingapplied to the work 10. Also, the trajectory planning unit 750 generatesthe trajectory information prescribing the above-described interpolationtrajectory. The trajectory information may be information indicating anangle of each joint of the robot arm 414 at each time.

On the other hand, when the trajectory planning unit 750 determines thatthe interference occurs on the calculated trajectory, the trajectoryplanning unit 750 accesses to the trajectory candidate extracting unit740 to request for information indicating a combination having anevaluation that is next highest after the previous combination. As thetrajectory planning unit 750 receives the information indicating a newcombination from the trajectory candidate extracting unit 740, thetrajectory planning unit 750 generates the trajectory informationaccording to procedures similar to the above-described procedures. Thecombination having the next highest evaluation may be one example ofanother combination.

In a case in which the amount or duration time of calculation in thetrajectory planning unit 750 satisfies the predetermined condition, thetrajectory planning unit 750 suspends the process regarding the currentcombination. For example, the trajectory planning unit 750 determinesthe amount or duration time of calculation allocated to theabove-described determining processing so as to generate the trajectoryinformation within a range of the constraint condition regarding theamount or duration time of calculation set by the concurrent processingcontrol unit 550. If the determining processing is not ended even whenthe determined amount or duration time of calculation is exceeded, thetrajectory planning unit 750 suspends the determining processingregarding the current combination. After that, the trajectory planningunit 750 accesses to the trajectory candidate extracting unit 740 torequest for information indicating a combination having an evaluationthat is next highest after the current combination. As the trajectoryplanning unit 750 receives the information indicating a new combinationfrom the trajectory candidate extracting unit 740, the trajectoryplanning unit 750 generates the trajectory information according toprocedures similar to the above-described procedures.

In the present embodiment, the control information generating unit 760generates the control information for controlling the picking hand 412and the robot arm 414. For example, the control information generatingunit 760 receives the trajectory information from the trajectoryplanning unit 750. For example, the control information generating unit760 adds, to the trajectory information generated by the trajectoryplanning unit 750, the another information that is necessary forcontrolling the picking hand 412 and the robot arm 414 to generate thecontrol information. The control information generating unit 760 mayalso convert the trajectory information generated by the trajectoryplanning unit 750 into a format in accordance with the type of the robot140 to generate the control information. The control informationgenerating unit 760 sends the control information to the execution unit540.

FIG. 8 schematically shows another example of the internal configurationof the execution unit 540. In the present embodiment, the execution unit540 includes a control information sending unit 810 and an executionmonitoring unit 820. The control information sending unit 810 may be oneexample of the instruction sending unit. The execution monitoring unit820 may be one example of the control device, the point cloudinformation acquiring unit, the abnormality detection unit and theinstruction sending unit.

In the present embodiment, the control information sending unit 810sends various control information to each unit of the distributionmanagement system 100 to control at least some of the distributionmanagement system 100. The control information sending unit 810 may alsocontrol the transfer apparatus 130 or the transfer control apparatus160. For example, the control information sending unit 810 sends thecontrol information for controlling the robot 140 to the drive controlunit 430 to control the operation of the robot 140. If the controlinformation sending unit 810 receives, from the execution monitoringunit 820, a detection signal indicating that an abnormality is detected,the control information sending unit 810 may also send an instructionfor suspending the operation of the robot 140 to the robot 140.

The control information sending unit 810 may also control the imagecapturing apparatus 152 and the image capturing apparatus 154. Forexample, the control information sending unit 810 may control aposition, an angle, magnification and the like of an image capturingapparatus for each image capturing apparatus.

The control information sending unit 810 may also control at least oneof the recognition unit 520 and the planning unit 530. For example, ifan abnormality occurs on the robot 140, the control information sendingunit 810 suspends the current process and sends an instruction forstarting new recognition processing and planning processing to therecognition unit 520 and the planning unit 530. The control informationsending unit 810 requests, at a specific time, the point cloudinformation generating unit 620 to send point cloud information of atleast one of the supply pallet 12 and the delivery pallet 14. The supplypallet 12 and the delivery pallet 14 may be one example of a regionincluding at least one of the picking hand 412 and the work 10.

As the above-described specific time, at least one of the followings canbe illustrated: (i) a time at which the picking hand 412 picks the work10 at the holding position, (ii) a time at which the picking hand 412moves the work 10 out of the supply pallet 12, (iii) a time at which thepicking hand 412 moves the work 10 in the delivery pallet 14, (iv) atime at which the picking hand 412 arranges the work 10 at thearrangement position. Accordingly, an abnormality such as aninterference and load collapse can be detected in an early stage.

In the present embodiment, the execution monitoring unit 820 monitorsthe operation of the robot 140, the supply pallet 12 and the deliverypallet 14. For example, the execution monitoring unit 820 receives, fromthe point cloud information generating unit 620, the point cloudinformation of at least one of the supply pallet 12 and the deliverypallet 14 at the above-described specific time. The execution monitoringunit 820 detects the abnormality based on the above-described pointcloud information. The execution monitoring unit 820 may also detect theabnormality based on (i) the above-described point cloud information and(ii) at least one selected from a group including: a measurement valuefor a sensor arranged on the robot 140; a control value for at least oneof the picking hand 412 and the robot arm 414; and at least one oftorque, a voltage value and a current value for the driving unit 420.

If the abnormality is detected, the execution monitoring unit 820 sendsa detection signal indicating that the abnormality is detected to thecontrol information sending unit 810, for example. If the abnormality isdetected, the execution monitoring unit 820 may also send an instructionfor suspending the operation of the robot 140 to the robot 140.

FIG. 9 schematically shows one example of information processing in thetransfer control apparatus 160. According to the present embodiment,first, in a step 902 (the term “step” may be abbreviated as the letter“S”), the recognition unit 520 receives user order information orworking information from the article management apparatus 110. Therecognition unit 520 analyzes the user order information or the workinginformation. For example, the recognition unit 520 determines thepicking sequence of one or more articles to the transfer apparatus 130based on the user order information or the working information. Therecognition unit 520 may generate information in which theidentification information of the one or more articles are arranged inthe picking sequence to the transfer apparatus 130.

In S912, the recognition unit 520 performs recognition processing. InS914, the planning unit 530 performs planning processing. In S916, theexecution unit 540 performs execution processing. Also, the executionunit 540 monitors the operation of the robot 140 and the supply pallet12 and the delivery pallet 14 during a period in which the robot 140transfers the work 10. When occurrence of the abnormality is detected,the execution unit 540 suspends the processing in S916 and starts theprocessing from S912 again.

Next, in S920, the execution unit 540 determines whether or not thetransfer processing of all articles included in the user orderinformation or working information is ended, for example. If theexecution unit 540 determines that the transfer processing of allarticles is ended (in a case of “Yes” in S920), the execution unit 540ends the processing. On the other hand, if the execution unit 540determines that the transfer processing of all articles is not ended (ina case of “No” in S920), the execution unit 540 performs, on a nextarticle, processing from S912 to S920.

FIG. 10 schematically shows one example of the recognition processing inS912. In the present embodiment, (i) the process of generating the pointcloud information and the position and posture information regarding thesupply pallet 12 and (ii) the process of generating the point cloudinformation and the position and posture information regarding thedelivery pallet 14 are performed concurrently or in parallel. In thepresent embodiment, the process of generating the point cloudinformation and the process of generating the position and postureinformation are performed concurrently or in parallel.

According to the present embodiment, in S1022, the position and postureinformation generating unit 610 acquires the image data of the supplypallet 12 from the image capturing apparatus 152. Also, the point cloudinformation generating unit 620 acquires the point cloud data of thesupply pallet 12 from the image capturing apparatus 152. The point cloudinformation generating unit 620 may also generate the point cloud dataof the supply pallet 12 based on the image data of the supply pallet 12.

Next, in S1024, the point cloud information generating unit 620generates the point cloud information of the supply pallet 12. Also, inS1026, the position and posture information generating unit 610generates the position and posture information of the supply pallet 12and the article stored in the supply pallet 12. In the presentembodiment, S1024 and S1026 are performed concurrently or in parallel.Note that during a period in which S1026 is performed for one time,S1024 may be performed for multiple times.

On the other hand, in S1042, the position and posture informationgenerating unit 610 acquires the image data of the delivery pallet 14from the image capturing apparatus 154. Also, the point cloudinformation generating unit 620 acquires the point cloud data of thedelivery pallet 14 from the image capturing apparatus 154. The pointcloud information generating unit 620 may also generate the point clouddata of the delivery pallet 14 based on the image data of the deliverypallet 14. In the present embodiment, S1022 and S1042 are performedconcurrently or in parallel.

Next, in S1044, the point cloud information generating unit 620generates the point cloud information of the delivery pallet 14. Also,in S1046, the position and posture information generating unit 610generates the position and posture information of the delivery pallet 14and the article stored in the delivery pallet 14. In the presentembodiment, S1044 and S1046 are performed concurrently or in parallel.Note that during a period in which S1046 is performed for one time,S1044 may also be performed for multiple times.

FIG. 11 schematically shows one example of the planning processing inS914. In the present embodiment, the process of extracting the holdingcandidate in the holding planning unit 710 and the process of extractingthe arrangement candidate in the arrangement planning unit 720 areperformed concurrently or in parallel. According to the presentembodiment, in S1120, the holding planning unit 710 extracts the holdingcandidate. Also, in S1140, the arrangement planning unit 720 extractsthe arrangement candidate. In the present embodiment, S1120 and S1140are performed concurrently or in parallel. After that, in S1160, thetrajectory candidate extracting unit 740 extracts the trajectorycandidate and the trajectory planning unit 750 determines thetrajectory. Also, in S1180, an interpolation trajectory of thedetermined trajectory is generated.

FIG. 12 schematically shows one example of the trajectory planningprocessing in S1160. In the present embodiment, the processes in whichthe trajectory candidate extracting unit 740 calculates the evaluationfunctions for at least some of M×N combinations regarding M holdingcandidates and N arrangement candidates are performed concurrently or inparallel. According to the present embodiment, in S1202, the trajectorycandidate extracting unit 740 calculates the evaluation function foreach of the M×N combinations regarding the M holding candidates and theN arrangement candidates. At this time, the trajectory candidateextracting unit 740 performs the process of calculating the evaluationfunction for each of the M×N combinations concurrently or in parallel.After that, in S1204, the trajectory candidate extracting unit 740extracts one combination from among the M×N combinations based on theevaluation function of each combination.

Next, in S1206, the trajectory planning unit 750 calculates a trajectoryon which (i) the picking hand 412 picks, at a holding position, a workwith a posture indicated by the holding candidate included in thecombination extracted by the trajectory candidate extracting unit 740and (ii) the picking hand 412 arranges, at an arrangement position, thework with a posture indicated by the arrangement candidate included inthe combination extracted by the trajectory candidate extracting unit740.

In S1208, the trajectory planning unit 750 determines whether theinterference is present or absent on the calculated trajectory. In thepresent embodiment, the trajectory planning unit 750 divides thecalculated trajectory into a plurality of zones to determine whether theinterference is present or absent for each zone. If it is determinedthat no interference occurs on the trajectory in all zones (in a case of“Yes” in S1208), the trajectory planning unit 750 ends the processing.Note that even in this case, the processing from S1204 to S1208 may alsobe repeated within a range of the constraint condition regarding theamount or duration time of calculation set by the concurrent processingcontrol unit 550. On the other hand, if it is determined that aninterference occurs on the trajectory in at least some zones (in a caseof “No” in S1208), the processing from S1204 to S1208 is repeated.

FIG. 13 schematically shows one example of the interference checkingprocessing in S1208. In the present embodiment, the trajectory planningunit 750 divides the trajectory calculated in S1206 into a plurality ofzones to determine whether an interference is present or absent for eachzone. Also, the trajectory planning unit 750 performs the determiningprocessing in at least two zones concurrently or in parallel. Accordingto the present embodiment, in S1302, the trajectory planning unit 750determines whether the interference is present or absent on thetrajectory calculated in S1206. The trajectory planning unit 750 dividesthe trajectory calculated in S1206 into a plurality of zones todetermine whether the interference is present or absent for each zone.The trajectory planning unit 750 performs the determining processing inall zones concurrently or in parallel, for example.

In the determining processing for each zone, if the duration time oramount of calculation exceeds a predetermined threshold before thedetermining processing is ended (in a case of “Yes” in S1304), thetrajectory planning unit 750 cancels the determining processing in allzones and returns to the processing in S1204. On the other hand, if thedetermining processing is ended before the duration time or amount ofcalculation exceeds the predetermined threshold and when it isdetermined that an interference occurs in one zone (in a case of “Yes”in S1306), the trajectory planning unit 750 cancels the determiningprocessing in all zones and returns to the processing in S1204. On theother hand, if the determining processing is ended before the durationtime or amount of calculation exceeds the predetermined threshold andwhen it is determined that an interference does not occur in the zone(in a case of “No” in S1306), the determining processing in the zone isended.

Concurrent processing between works or between orders in the transfercontrol apparatus 160 is described by using FIG. 14, FIG. 15 and FIG.16. FIG. 14 schematically shows one example of the data table 1400. FIG.15 schematically shows one example of the concurrent processing betweenworks. FIG. 16 schematically shows one example of the concurrentprocessing between works.

As shown in FIG. 14, in the present embodiment, the data table 1400stores a work ID 1410, an order ID 1420 and contents 1430 of an order inassociation with each other. The contents 1430 of the order include anarticle ID 1432 for identifying an article to be a transfer target, asupply pallet ID 1434 for identifying a supply pallet, and a deliverypallet ID 1436 for identifying a delivery pallet, for example.

As shown in FIG. 15, according to the present embodiment, the planningunit 530 performs, after performing planning processing of a work havingthe work ID 1410 being W_0001 and before completing execution processingof the work having the work ID 1410 being W_0001, planning processing ofthe work having the work ID 1410 being W_0002 and recognition processingof the work having the work ID 1410 being W_0003. Accordingly, duringthe period in which the execution processing of the work having the workID 1410 being W_0001 is performed, the planning processing of the workhaving the work ID 1410 being W_0002 is performed and the recognitionprocessing of the work having the work ID 1410 being W_0003 isperformed.

The amount or duration time of calculation of the planning processing ofthe work having the work ID 1410 being W_0002 is adjusted such that theplanning processing is ended before the execution processing of the workhaving the work ID 1410 being W_0001 is completed, for example. Morespecifically, the concurrent processing control unit 550 adjusts theamount or duration time of calculation of the planning unit 530according to the following procedures.

For example, the concurrent processing control unit 550 estimates aperiod of time that is from the time when the robot 140 receives thecontrol information regarding a work having the work ID 1410 beingW_0001 to the time when the operation of the robot 140 on the work isended. The concurrent processing control unit 550 may also estimate theabove-described period of time based on the trajectory information ofthe work having the work ID 1410 being W_0001. The above-describedperiod of time may be one example of the constraint condition. Theconcurrent processing control unit 550 sends the above-describedestimation result to the planning unit 530.

On the other hand, the planning unit 530 adjusts the amount ofcalculation of the work planning processing such that the planningprocessing of the work having the work ID 1410 being W_0002 is completedbefore the period of time estimated by the concurrent processing controlunit 550 is expired or elapsed. For example, the amount of calculationof the planning processing is adjusted by adjusting a value of thethreshold in S1304.

As shown in FIG. 16, in the recognition processing of the work havingthe work ID 1410 being W_0001, the recognition processing of the supplypallet 12 having a supply pallet ID 1434 being S_0001 and therecognition processing of the delivery pallet 14 having a deliverypallet ID 1436 being D_0001 are performed concurrently or in parallel.Also, in the recognition processing of each supply pallet 12, theprocess of generating the point cloud information and the process ofgenerating the position and posture information are performedconcurrently or in parallel. Similarly, in the recognition processing ofeach delivery pallet 14, the process of generating the point cloudinformation and the process of generating the position and postureinformation are performed concurrently or in parallel.

In the present embodiment, the process of generating the point cloudinformation is performed for multiple times during the period in whichthe recognition processing is performed for one time. Accordingly, therecognition unit 520 can provide the point cloud information to anotherelement (for example, the execution unit 540) at a proper time. Also,waiting time in another element until the point cloud information isprovided is reduced.

While the embodiments of the present invention have been described, thetechnical scope of the invention is not limited to the above describedembodiments. It is apparent to persons skilled in the art that variousalterations and improvements can be added to the above-describedembodiments. Also, the matters described with reference to a specificembodiment can be applied to another embodiment as long as suchapplication does not cause a technical contradiction. Also, eachcomponent may also have a similar feature to another component that hasthe same name but has a different reference sign. It is also apparentfrom the scope of the claims that the embodiments added with suchalterations or improvements can be included in the technical scope ofthe invention.

The operations, procedures, steps, and stages of each process performedby an apparatus, system, program, and method shown in the claims,embodiments, or diagrams can be performed in any order as long as theorder is not indicated by “prior to,” “before,” or the like and as longas the output from a previous process is not used in a later process.Even if the process flow is described using phrases such as “first” or“next” in the claims, embodiments, or diagrams, it does not necessarilymean that the process must be performed in this order.

For example, in the specification of the present application, thefollowing matters are described.

[Item 1-1]

A control device for controlling a picking apparatus that arranges, at asecond position, a work that is picked at a first position by using apicking hand, the second position being different from theabove-described first position, the control device including:

a first candidate information acquiring unit for acquiring firstcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described firstposition;

a second candidate information acquiring unit for acquiring secondcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described secondposition;

an extracting unit for extracting one combination from among one or morecombinations regarding one or more postures indicated by theabove-described first candidate information and one or more posturesindicated by the above-described second candidate information;

a trajectory calculating unit for calculating a trajectory on which (i)the above-described picking hand picks, at the above-described firstposition, the above-described work with a posture associated with theabove-described first position included in the above-described onecombination and (ii) the above-described picking hand arranges, at theabove-described second position, the above-described work with a postureassociated with the above-described second position included in theabove-described one combination; and

a determining unit for determining whether an interference is present orabsent on the trajectory calculated by the above-described trajectorycalculating unit.

[Item 1-2]

The control device according to Item 1-1, wherein

when the above-described determining unit determines that theinterference occurs on the above-described trajectory,

the above-described extracting unit extracts another combination fromamong the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectoryon which (i) the above-described picking hand picks, at theabove-described first position, the above-described work with a postureassociated with the above-described first position included in theabove-described other combination and (ii) the above-described pickinghand arranges, at the above-described second position, theabove-described work with a posture associated with the above-describedsecond position included in the above-described other combination, and

the above-described determining unit determines whether the interferenceis present or absent on the trajectory being calculated by theabove-described trajectory calculating unit and being associated withthe above-described other combination.

[Item 1-3]

The control device according to Item 1-1, wherein

when an amount or duration time of calculation in at least one of theabove-described extracting unit, the above-described trajectorycalculating unit and the above-described determining unit satisfies apredetermined condition,

the above-described extracting unit extracts another combination fromamong the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectoryon which (i) the above-described picking hand picks, at theabove-described first position, the above-described work with a postureassociated with the above-described first position included in theabove-described other combination and (ii) the above-described pickinghand arranges, at the above-described second position, theabove-described work with a posture associated with the above-describedsecond position included in the above-described other combination, and

the above-described determining unit determines whether the interferenceis present or absent on the trajectory being calculated by theabove-described trajectory calculating unit and being associated withthe above-described other combination.

[Item 1-4]

The control device according to any one of Items 1-1 to 1-3, wherein

the above-described trajectory calculating unit calculates a trajectoryon which (i) the above-described picking hand moves from a thirdposition that is a position of the above-described picking hand when theabove-described picking apparatus starts working on the above-describedwork to the above-described first position, (ii) the above-describedpicking hand picks, at the above-described first position, theabove-described work with a posture associated with the above-describedfirst position included in the combination extracted by theabove-described extracting unit and (iii) the above-described pickinghand arranges, at the above-described second position, theabove-described work with a posture associated with the above-describedsecond position included in the combination extracted by theabove-described extracting unit.

[Item 1-5]

The control device according to any one of Items 1-1 to 1-4, wherein

the above-described extracting unit calculates an evaluation functionfor at least some of the above-described one or more combinations, and

selects the above-described one combination based on the above-describedevaluation function.

[Item 1-6]

The control device according to Item 1-5, wherein

the above-described evaluation function includes an evaluation itemregarding an estimated value of a period of time that is from (i) a timewhen the above-described picking hand picks, at the above-describedfirst position, the above-described work with the posture associatedwith the above-described first position included in a combination thatis to be an evaluation target by the above-described evaluation functionto (ii) a time when the above-described picking hand arranges, at theabove-described second position, the above-described work with theposture associated with the above-described second position included inthe combination that is to be the evaluation target by theabove-described evaluation function.

[Item 1-7]

The control device according to Item 1-5, wherein

the above-described evaluation function includes an evaluation itemregarding an estimated value of a period of time that is from (i) a timewhen the above-described picking hand moves from a third position thatis a position of the above-described picking hand when theabove-described picking apparatus starts working on the above-describedwork to the above-described first position, (ii) the above-describedpicking hand picks, at the above-described first position, theabove-described work with a posture associated with the above-describedfirst position included in the combination that is to be an evaluationtarget by the above-described evaluation function, to (iii) a time whenthe above-described picking hand arranges, at the above-described secondposition, the above-described work with a posture associated with theabove-described second position included in the combination that is tobe the evaluation target by the above-described evaluation function.

[Item 1-8]

The control device according to any one of Items 1-1 to 1-7, wherein

the above-described determining unit divides the trajectory calculatedby the above-described trajectory calculating unit into a plurality ofzones,

determines whether an interference is present or absent for each zone.

[Item 1-9]

The control device according to any one of Items 1-1 to 1-8, furtherincluding a trajectory determining unit for determining a trajectory ofthe above-described picking hand based on a determination result of theabove-described determining unit.

[Item 1-10]

The control device according to Item 1-9, wherein

the above-described trajectory determining unit has an interpolationtrajectory generating unit for generating an interpolation trajectory ofthe trajectory calculated by the above-described trajectory calculatingunit.

[Item 1-11]

A picking system including the control device according to any one ofItems 1-1 to 1-10; and

the above-described picking apparatus.

[Item 1-12]

A distribution system including

the control device according to any one of Items 1-1 to 1-10;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 1-13]

A program to cause a computer to serve as the control device accordingto any one of Items 1-1 to 1-10.

[Item 1-14]

A control method for controlling a picking apparatus that arranges, at asecond position, a work picked at a first position by using a pickinghand, the second position being different from the above-described firstposition, the control method including:

a first candidate information acquiring step of acquiring firstcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described firstposition;

a second candidate information acquiring step of acquiring secondcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described secondposition;

an extracting step of extracting one combination from among one or morecombinations regarding one or more postures indicated by theabove-described first candidate information and one or more posturesindicated by the above-described second candidate information;

a trajectory calculating step of calculating a trajectory on which (i)the above-described picking hand picks, at the above-described firstposition, the above-described work with a posture associated with theabove-described first position included in the above-described onecombination and (ii) the above-described picking hand arranges, at theabove-described second position, the above-described work with a postureassociated with the above-described second position included in theabove-described one combination; and

a determining step of determining whether an interference is present orabsent on the trajectory calculated in the above-described trajectorycalculating step.

[Item 1-15]

A method of producing a container in which a work is arranged by using apicking apparatus which arranges, at a second position, theabove-described work picked at a first position by using a picking hand,the second position being different from the above-described firstposition, the method of producing the container including:

a step of setting a specific position of the above-described containerto the above-described second position;

a first candidate information acquiring step of acquiring firstcandidate information indicating each of one or more postures thatpicking hand takes on the above-described first position;

a second candidate information acquiring step of acquiring secondcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described secondposition;

an extracting step of extracting one combination from among one or morecombinations regarding one or more postures indicated by theabove-described first candidate information and one or more posturesindicated by the above-described second candidate information;

a trajectory calculating step of calculating a trajectory on which (i)the above-described picking hand picks, at the above-described firstposition, the above-described work with a posture associated with theabove-described first position included in the above-described onecombination and (ii) the above-described picking hand arranges, at theabove-described second position, the above-described work with a postureassociated with the above-described second position included in theabove-described one combination; and

a determining step of determining whether an interference is present orabsent on the trajectory calculated in the above-described calculatingthe trajectory.

[Item 2-1]

A control device for controlling a picking apparatus that arranges, at asecond position of a second container, a work picked at a first positionof a first container by using a picking hand, the second container beingdifferent from the above-described first container, the control deviceincluding:

a trajectory information generating unit for generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks the above-described work at the above-described firstposition and arranges the work at the above-described second position;

an execution control unit for operating the above-described pickingapparatus based on the trajectory information generated by theabove-described trajectory information generating unit; and

an execution time estimating unit for estimating a period of time thatis from the time when the above-described picking apparatus receives aninstruction for starting an operation on the above-described work to thetime when the operation of the above-described picking apparatus on theabove-described work is ended, wherein

the above-described trajectory information generating unit adjusts anamount of calculation based on an estimation result of theabove-described execution time estimating unit.

[Item 2-2]

The control device according to Item 2-1, wherein

after the above-described trajectory information generating unitperforms process of generating trajectory information regarding a firstwork and before a movement of the above-described first work iscompleted, the above-described trajectory information generating unitperforms the process of generating trajectory information regarding asecond work being different from the above-described first work,

the above-described execution time estimating unit estimates, based onthe trajectory information being generated by the above-describedtrajectory information generating unit and regarding the above-describedfirst work, a first period of time that is from the time when theabove-described picking apparatus receives an instruction for startingan operation on the above-described first work to the time when anoperation of the above-described picking apparatus on theabove-described first work is ended, and

before the above-described first period of time estimated by theabove-described execution time estimating unit is expired or elapsed,the above-described trajectory information generating unit adjusts anamount of calculation of a process of generating trajectory informationregarding the above-described second work such that a process ofgenerating trajectory information regarding the above-described secondwork is completed.

[Item 2-3]

The control device according to Item 2-2, wherein

the above-described trajectory information generating unit generates thetrajectory information regarding the above-described second work byusing an operation result in the process of generating the trajectoryinformation regarding the above-described first work.

[Item 2-4]

The control device according to any one of Items 2-1 to 2-3, wherein

the above-described trajectory information generating unit includes:

a first candidate information acquiring unit for acquiring firstcandidate information indicating each of one or more postures thatpicking hand takes on the above-described first position;

a second candidate information acquiring unit for acquiring secondcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described secondposition;

an extracting unit for extracting one combination from among one or morecombinations regarding one or more postures indicated by theabove-described first candidate information and one or more posturesindicated by the above-described second candidate information;

a trajectory calculating unit for calculating a trajectory on which (i)the above-described picking hand picks, at the above-described firstposition, the above-described work with a posture associated with theabove-described first position included in the above-described onecombination and (ii) the above-described picking hand arranges, at theabove-described second position, the above-described work with a postureassociated with the above-described second position included in theabove-described one combination; and

a determining unit for determining whether an interference is present orabsent on the trajectory calculated by the above-described trajectorycalculating unit.

[Item 2-5]

The control device according to Item 2-4, wherein

when the above-described determining unit determines that theinterference occurs on the above-described trajectory,

the above-described extracting unit extracts another combination fromamong the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectoryon which (i) the above-described picking hand picks, at theabove-described first position, the above-described work with a postureassociated with the above-described first position included in theabove-described other combination and (ii) above-described picking handarranges, at the above-described second position, the above-describedwork with a posture associated with the above-described second positionincluded in the above-described other combination, and

the above-described determining unit determines whether the interferenceis present or absent on the trajectory being calculated by theabove-described trajectory calculating unit and being associated withthe above-described other combination.

[Item 2-6]

The control device according to Item 2-4, wherein

when an amount or duration time of calculation in at least one of theabove-described extracting unit, the above-described trajectorycalculating unit and the above-described determining unit satisfies apredetermined condition,

the above-described extracting unit extracts another combination fromamong the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectoryon which (i) the above-described picking hand picks, at theabove-described first position, the above-described work with a postureassociated with the above-described first position included in theabove-described other combination and (ii) the above-described pickinghand arranges, at the above-described second position, theabove-described work with a posture associated with the above-describedsecond position included in the above-described other combination, and

the above-described determining unit determines whether the interferenceis present or absent on the trajectory being calculated by theabove-described trajectory calculating unit and being associated withthe above-described other combination.

[Item 2-7]

The control device according to any one of Items 2-4 to 2-6, furtherincluding a constraint condition setting unit for determining aconstraint condition regarding an amount or duration time of calculationof processing in at least one of the above-described extracting unit,the above-described trajectory calculating unit and the above-describeddetermining unit based on an estimation result of the above-describedexecution time estimating unit, wherein

the above-described at least one of the above-described extracting unit,the above-described trajectory calculating unit and the above-describeddetermining unit adjusts an amount or duration time of calculation basedon the constraint condition set by the above-described constraintcondition setting unit.

[Item 2-8]

The control device according to Item 2-7, wherein

the above-described constraint condition is at least one of (i) themaximum value of the amount or duration time of calculation that isuntil a progress of the processing reaches a preset milestone and (ii)the maximum value of the amount or duration time of calculation that isuntil the processing is completed.

[Item 2-9]

The control device according to any one of Items 2-4 to 2-8, wherein

the above-described extracting unit calculates an evaluation functionfor at least some of the above-described one or more combinations, and

selects the above-described one combination based on the above-describedevaluation function.

[Item 2-10]

The control device according to any one of Items 2-4 to 2-9, wherein

the above-described determining unit divides the trajectory calculatedby the above-described trajectory calculating unit into a plurality ofzones, and

determines whether an interference is present or absent for each zone.

[Item 2-11]

The control device according to any one of Items 2-4 to 2-10, wherein

the above-described trajectory information generating unit further has atrajectory determining unit for determining a trajectory of theabove-described picking hand based on the determination result of theabove-described determining unit.

[Item 2-12]

A picking system including

the control device according to any one of Items 2-1 to 2-11; and

the above-described picking apparatus.

[Item 2-13]

A distribution system including

the control device according to any one of Items 2-1 to 2-10;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 2-14]

A program to cause a computer to serve as the control device accordingto any one of Items 2-1 to 2-11.

[Item 2-15]

A control method for controlling a picking apparatus that arranges, at asecond position of a second container, a work picked at a first positionof a first container by using a picking hand, the second container beingdifferent from the above-described first container, the control methodincluding:

a trajectory information generating step of generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks the work at the above-described first position andarranges the work at the above-described second position;

an execution control step of operating the above-described pickingapparatus based on the trajectory information generated in theabove-described trajectory information generating step; and

an execution time estimating step of estimating a period of time that isfrom the time when the above-described picking apparatus receives aninstruction for starting an operation on the above-described work in theabove-described execution control step to the time when the operation ofthe above-described picking apparatus on the above-described work isended, wherein

the above-described trajectory information generating step includes astep of adjusting the amount of calculation based on an estimationresult in the above-described execution time estimating step.

[Item 2-16]

A production method of producing a second container in which a work isarranged by using a picking apparatus that arranges, at a secondposition of a second container, the above-described work picked at afirst position of a first container by using a picking hand, theabove-described second container being different from theabove-described first container, the production method including:

a trajectory information generating step of generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks the work at the above-described first position andarranges the work at the above-described second position;

an execution control step of operating the above-described pickingapparatus based on trajectory information generated in theabove-described trajectory information generating step;

an execution time estimating step of estimating a period of time that isfrom the time when the above-described picking apparatus receives aninstruction for starting an operation on the above-described work in theabove-described execution control step to the time when the operation ofthe above-described picking apparatus on the above-described work isended, wherein

the above-described trajectory information generating step includes astep of adjusting an amount of calculation based on an estimation resultin the above-described execution time estimating step.

[Item 3-1]

A control device for controlling a picking apparatus that arranges, at asecond position of a second container, a work picked at a first positionof a first container by using a picking hand, the second container beingdifferent from the above-described first container, the control deviceincluding:

a point cloud information acquiring unit for acquiring point cloudinformation of a region including at least one of the above-describedpicking hand and the above-described work at a time being at least oneof (i) when the above-described picking hand picks the above-describedwork at the above-described first position, (ii) when theabove-described picking hand moves the above-described work out of theabove-described first container, (iii) when the above-described pickinghand moves the above-described work in the above-described secondcontainer and (iv) when the above-described picking hand arranges theabove-described work at the above-described second position; and

an abnormality detection unit for detecting an abnormality based on thepoint cloud information acquired by the above-described point cloudinformation acquiring unit.

[Item 3-2]

The control device according to Item 3-1, further including an imageacquiring unit for acquiring image data of an image including at leastone of the above-described picking hand and the above-described work ina field angle at the above-described at least one time, wherein

the above-described point cloud information acquiring unit acquires theabove-described point cloud information based on the image acquired bythe above-described image acquiring unit.

[Item 3-3]

The control device according to Item 3-1 or 3-2, wherein

the above-described point cloud information is three-dimensional pointcloud information.

[Item 3-4]

The control device according to any one of Items 3-1 to 3-3, wherein

the above-described abnormality detection unit is for detecting anabnormality based on (i) the above-described point cloud information and(ii) at least one selected from a group including: a measurement valueof a sensor arranged on the above-described picking apparatus; a controlvalue for at least one of the above-described picking hand, and a robotarm coupled with the above-described picking hand; and at least one oftorque, a voltage value and a current value for a driving unit thatdrives at least one of the above-described picking hand and theabove-described robot arm.

[Item 3-5]

The control device according to any one of Items 3-1 to 3-4, furtherincluding an instruction sending unit for sending, to theabove-described picking apparatus, an instruction for suspending anoperation of the above-described picking apparatus when theabove-described abnormality detection unit detects the abnormality.

[Item 3-6]

A picking system including

a control device according to any one of Items 3-1 to 3-5; and

the above-described picking apparatus.

[Item 3-7]

A distribution system including

the control device according to any one of Items 3-1 to 3-5;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 3-8]

A program for causing a computer to serve as the control deviceaccording to any one of Items 3-1 to 3-5.

[Item 3-9]

A control method for controlling a picking apparatus that arranges, at asecond position of a second container, a work picked at a first positionof a first container by using a picking hand, the second container beingdifferent from the above-described first container, the control methodincluding:

a point cloud information acquiring procedure of acquiring point cloudinformation of a region including at least one of the above-describedpicking hand and the above-described work at a time being at least oneof (i) when the above-described picking hand picks the above-describedwork at the above-described first position, (ii) when theabove-described picking hand moves the above-described work out of theabove-described first container, (iii) when the above-described pickinghand moves the above-described work in the above-described secondcontainer and (iv) when the above-described picking hand arranges theabove-described work at the above-described second position; and

an abnormality detection procedure of detecting an abnormality based onthe point cloud information acquired in the above-described point cloudinformation acquiring procedure.

[Item 4-1]

A control device for controlling a picking apparatus that moves one ormore works from one or more supply containers to one or more arrangementcontainers by using a picking hand, the control device including

a trajectory information generating unit for generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks one of the above-described one or more works storedin one supply container included in the above-described one or moresupply containers and arranges the work on one arrangement containerincluded in the above-described one or more arrangement containers,wherein

after the process of generating trajectory information regarding a firstwork included in the above-described one or more work is performed andbefore the movement of the above-described first work is completed,

the above-described trajectory information generating unit generatestrajectory information regarding a second work being different from theabove-described first work and included in the above-described one ormore work by using an operation result in the process for generating thetrajectory information regarding the above-described first work.

[Item 4-2]

The control device according to Item 4-1, further including a containerinformation acquiring unit for acquiring information regarding at leastone of a state of a surface of the above-described one supply containerand a state of a surface of the above-described one arrangementcontainer, wherein

the above-described trajectory information generating unit (a) acquiresor generates information regarding a position and a posture of theabove-described first work in the above-described one supply containerbefore the above-described picking apparatus moves the above-describedfirst work in processing for generating the trajectory informationregarding the above-described first work, and

(b) generates trajectory information regarding the above-describedsecond work by using (i) the information acquired by the above-describedcontainer information acquiring unit before the movement of theabove-described first work is completed and (ii) the informationregarding the position and the posture of the above-described first workin the above-described one supply container in the process of generatingthe trajectory information regarding the above-described second work.

[Item 4-3]

The control device according to Item 4-1, further including a containerinformation acquiring unit for acquiring information regarding a stateof a surface of at least one of the above-described one supply containerand the above-described one arrangement container, wherein

the above-described trajectory information generating unit (a) acquiresor generates information regarding a position and a posture of theabove-described first work in the above-described one arrangementcontainer after the above-described picking apparatus moves theabove-described first work in the process of generating the trajectoryinformation regarding the above-described first work, and

(b) generates trajectory information regarding the above-describedsecond work by using (i) information acquired by the above-describedcontainer information acquiring unit before the movement of theabove-described first work is completed and (ii) information regarding aposition and a posture of the above-described first work on theabove-described one arrangement container in the process of generatingthe trajectory information regarding the above-described second work.

[Item 4-4]

The control device according to any one of Items 4-1 to 4-3, furtherincluding an order information acquiring unit for acquiring (i) firstorder information that instructs to move the above-described one or moreworks from at least one of the above-described one or more supplycontainers to a first arrangement container included in theabove-described one or more arrangement containers and (ii) second orderinformation that instruct to move the above-described one or more worksfrom at least one of the above-described one or more supply containersto a second arrangement container being included in the above-describedone or more arrangement containers and being different from theabove-described first arrangement container, wherein the above-describedtrajectory information generating unit performs a process of generatingtrajectory information regarding the above-described first work inaccordance with the acquisition of one of the above-described firstorder information and the above-described second order information bythe above-described order information acquiring unit, and

performs the process of generating trajectory information regarding theabove-described second work in accordance with the acquisition of theother of the above-described first order information and theabove-described second order information by the above-described orderinformation acquiring unit.

[Item 4-5]

The control device according to Item 4-4, wherein

the above-described first order information instructs to move theabove-described first work from a first supply container included in theabove-described one or more supply containers to the above-describedfirst arrangement container,

the above-described second order information instructs to move theabove-described second work from the above-described first supplycontainer to the above-described second arrangement container, and

the above-described trajectory information generating unit performs theprocess of generating trajectory information regarding theabove-described first work in accordance with the acquisition of theabove-described first order information by the above-described orderinformation acquiring unit, and

performs the process of generating the trajectory information regardingthe above-described second work in accordance with the acquisition ofthe above-described second order information by the above-describedorder information acquiring unit.

[Item 4-6]

The control device according to any one of Items 4-1 to 4-3, furtherincluding an order information acquiring unit for acquiring (i) firstorder information that instructs to move the above-described one or moreworks from a first supply container included in the above-described oneor more supply containers to at least one of the above-described one ormore arrangement containers and (ii) second order information thatinstructs to move the above-described one or more works from a secondsupply container being included in the above-described one or moresupply containers and being different from the above-described firstsupply container to at least one of the above-described one or morearrangement containers, wherein

the above-described trajectory information generating unit performs theprocess of generating trajectory information regarding theabove-described first work in accordance with the acquisition of one ofthe above-described first order information and the above-describedsecond order information by the above-described order informationacquiring unit, and

performs the process of generating trajectory information regarding theabove-described second work in accordance with the acquisition of theother of the above-described first order information and theabove-described second order information by the above-described orderinformation acquiring unit.

[Item 4-7]

The control device according to Item 4-6, wherein

the above-described first order information instructs to move theabove-described first work from the above-described first supplycontainer to a first arrangement container included in theabove-described one or more supply containers,

the above-described second order information instructs to move theabove-described second work from the above-described second supplycontainer to the above-described first arrangement container, and

the above-described trajectory information generating unit performs theprocess of generating the trajectory information regarding theabove-described first work in accordance with the acquisition of theabove-described first order information by the above-described orderinformation acquiring unit, and

performs the process of generating trajectory information regarding theabove-described second work in accordance with the acquisition of theabove-described second order information by the above-described orderinformation acquiring unit.

[Item 4-8]

The control device according to any one of Items 4-1 to 4-3, furtherincluding an order information acquiring unit for acquiring (i) firstorder information that instructs to move the above-described first workfrom first supply container included in the above-described one or moresupply containers to a first arrangement container included in theabove-described one or more arrangement containers and (ii) second orderinformation that instructs to move the above-described second work froma second supply container being included in the above-described one ormore supply containers and being different from the above-describedfirst supply container to a second arrangement container being includedin the above-described one or more arrangement containers and beingdifferent from the above-described first arrangement container, and

the above-described trajectory information generating unit performs theprocess of generating trajectory information regarding theabove-described first work in accordance with the acquisition of theabove-described first order information by the above-described orderinformation acquiring unit, and performs the process of generating thetrajectory information regarding the above-described second work inaccordance with the acquisition of the above-described second orderinformation by the above-described order information acquiring unit.

[Item 4-9]

A picking system including

the control device according to any one of Items 4-1 to 4-8; and

the above-described picking apparatus.

[Item 4-10]

A distribution system including

the control device according to any one of Items 4-1 to 4-8;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described one or more works.

[Item 4-11]

A program for causing a computer to serve as the control deviceaccording to any one of Items 4-1 to 4-8.

[Item 4-12]

A control method for controlling a picking apparatus that moves one ormore works from one or more supply containers to one or more arrangementcontainers by using a picking hand, the control method including

a trajectory information generating step of generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks one of the above-described one or more works storedin one supply container included in the above-described one or moresupply containers and arranges the work on one arrangement containerincluded in the above-described one or more arrangement containers,wherein

the above-described trajectory information generating step includes

a step of generating trajectory information regarding a second workbeing included in the above-described one or more works and beingdifferent from the above-described first work by using ab operationresult in the process of generating the trajectory information regardingthe above-described first work after performing the process ofgenerating the trajectory information regarding a first work included inthe above-described one or more works and before the movement of theabove-described first work is completed.

[Item 4-13]

A production method of producing one or more arrangement containers inwhich one or more works are arranged by using a picking apparatus thatmoves, by using a picking hand, the above-described one or more worksfrom one or more supply containers to the above-described one or morearrangement containers, the production method including

a trajectory information generating step of generating trajectoryinformation prescribing a trajectory on which the above-describedpicking hand picks one of the above-described one or more works storedin one supply container included in the above-described one or moresupply containers and arranges the work on one arrangement containerincluded in the above-described one or more arrangement containers,wherein

the above-described trajectory information generating step includes

a step of generating trajectory information regarding a second workbeing included in the above-described one or more works and beingdifferent from the above-described first work by using an operationresult in the process of generating the trajectory information regardingthe above-described first work after performing the process ofgenerating the trajectory information regarding a first work included inthe above-described one or more works and before the movement of theabove-described first work is completed.

[Item 5-1]

An information processing apparatus including:

a three-dimensional information acquiring unit for acquiringthree-dimensional information of a target object;

a holding information acquiring unit for acquiring holding informationthat defines a method of holding the above-described target object byusing a picking hand; and

an information management unit for storing, in a storage apparatus, (i)three-dimensional information of the above-described target objectacquired by the above-described three-dimensional information acquiringunit and (ii) the holding information acquired by the above-describedholding information acquiring unit in association with each other.

[Item 5-2]

The information processing apparatus according to Item 5-1, wherein

the three-dimensional information of the above-described target objectincludes at least one of home position information and coordinate axisinformation.

[Item 5-3]

The information processing apparatus according to Item 5-1 or 5-2,further including a two-dimensional information acquiring unit foracquiring two-dimensional information of the above-described targetobject, wherein

the above-described information management unit stores, in theabove-described storage apparatus, (i) three-dimensional information ofthe above-described target object acquired by the above-describedthree-dimensional information acquiring unit, (ii) holding informationacquired by the above-described holding information acquiring unit and(iii) two-dimensional information of the above-described target objectacquired by the above-described two-dimensional information acquiringunit in association with each other.

[Item 5-4]

The information processing apparatus according to Item 5-3, furtherincluding a feature information acquiring unit for acquiring featureinformation for recognizing the above-described target object based onthe two-dimensional information of the above-described target objectacquired by the above-described two-dimensional information acquiringunit, wherein

the above-described information management unit stores, in theabove-described storage apparatus, (i) three-dimensional information ofthe above-described target object acquired by the above-describedthree-dimensional information acquiring unit, (ii) holding informationacquired by the above-described holding information acquiring unit,(iii) two-dimensional information of the above-described target objectacquired by the above-described two-dimensional information acquiringunit and (iv) feature information of the above-described target objectacquired by the above-described feature information acquiring unit inassociation with each other.

[Item 5-5]

The information processing apparatus according to Item 5-1 or 5-2,further including:

a two-dimensional information acquiring unit for acquiringtwo-dimensional information of the above-described target object; and

a feature information acquiring unit for acquiring feature informationfor recognizing the above-described target object based ontwo-dimensional information of the above-described target objectacquired by the above-described two-dimensional information acquiringunit, wherein

the above-described information management unit stores, in theabove-described storage apparatus, (i) three-dimensional information ofthe above-described target object acquired by the above-describedthree-dimensional information acquiring unit, (ii) holding informationacquired by the above-described holding information acquiring unit and(iii) feature information of the above-described target object acquiredby the above-described feature information acquiring unit in associationwith each other.

[Item 5-6]

The information processing apparatus according to any one of Items 5-3to 5-5, wherein

the above-described two-dimensional information is image data of threeor more images of the above-described target object captured from threeor more directions.

[Item 5-7]

The information processing apparatus according to any one of Items 5-1to 5-6, further including the above-described storage apparatus.

[Item 5-8]

The information processing apparatus according to any one of Items 5-1to 5-7, wherein

the above-described information management unit stores, in a storageapparatus, (i) three-dimensional information of the above-describedtarget object acquired by the above-described three-dimensionalinformation acquiring unit, (ii) holding information acquired by theabove-described holding information acquiring unit and (iii)identification information for identifying the above-described targetobject in association with each other.

[Item 5-9]

The information processing apparatus according to Item 5-8, furtherincluding:

a picking control unit for controlling a picking apparatus thatarranges, at a second position, a work picked at a first position byusing a picking hand, the second position being different from theabove-described first position;

an order information acquiring unit for acquiring order informationincluding identification information of at least one work that is to bea working target of the picking apparatus;

a holding information storing unit for storing identificationinformation of one or more works and holding information of theabove-described one or more works in association with each other; and

a holding information extracting unit for (i) accessing to theabove-described storage apparatus and extract, from identificationinformation of the above-described at least one work included in orderinformation acquired by the above-described order information acquiringunit, the above-described holding information associated withidentification information of the above-described target object matchedwith identification information that is not stored in theabove-described holding information storing unit, and (ii) associatingthe identification information that is not stored in the above-describedholding information storing unit with the above-described extractedholding information and store the information in the above-describedholding information storing unit, wherein

the above-described holding information storing unit is realized by amedium being different from the above-described storage apparatus.

[Item 5-10]

The information processing apparatus according to Item 5-9, furtherincluding:

a first candidate information generating unit for generating firstcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described first positionwith reference to the holding information stored in the above-describedholding information storing unit;

a second candidate information generating unit for generating secondcandidate information indicating each of one or more postures that theabove-described picking hand takes on the above-described secondposition with reference to the holding information stored in theabove-described holding information storing unit;

an extracting unit for extracting one combination from among one or morecombinations regarding one or more postures indicated by theabove-described first candidate information and one or more posturesindicated by the above-described second candidate information;

a trajectory calculating unit for calculating a trajectory on which (i)the above-described picking hand picks, at the above-described firstposition, the above-described work with a posture associated with theabove-described first position included in the above-described onecombination and (ii) the above-described picking hand arranges, at theabove-described second position, the above-described work with a postureassociated with the above-described second position included in theabove-described one combination; and

a determining unit for determining whether an interference is present orabsent on the trajectory calculated by the above-described trajectorycalculating unit.

[Item 5-11]

A picking system including

the information processing apparatus according to Item 5-9 or 5-10; and

the above-described picking apparatus.

[Item 5-12]

A distribution system including

the information processing apparatus according to Item 5-9 or 5-10;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 5-13]

A program for causing a computer to serve as the information processingapparatus according to any one of Item 5-1 to 5-10.

[Item 5-14]

An information processing method including:

a three-dimensional information acquiring step of acquiring, by acomputer, three-dimensional information of a target object;

a holding information acquiring step of acquiring holding informationthat defines a method of holding, by a computer, the above-describedtarget object by using a picking hand; and

an information management step of storing, in a storage apparatus by acomputer, (i) three-dimensional information of the above-describedtarget object acquired by the above-described three-dimensionalinformation acquiring unit and (ii) holding information acquired by theabove-described holding information acquiring unit in association witheach other.

REFERENCE SIGNS LIST

10 . . . work; 12 . . . supply pallet; 14 . . . delivery pallet; 100 . .. distribution management system; 110 . . . article managementapparatus; 120 . . . data registration apparatus; 130 . . . transferapparatus; 140 . . . robot; 152 . . . image capturing apparatus; 154 . .. image capturing apparatus; 160 . . . transfer control apparatus; 170 .. . conveying apparatus; 172 . . . conveyor; 174 . . . conveyor; 210 . .. order information acquiring unit; 220 . . . conveying apparatusmanagement unit; 230 . . . transfer apparatus management unit; 240 . . .request processing unit; 250 . . . article information storing unit; 312. . . three-dimensional information acquiring unit; 314 . . .two-dimensional information acquiring unit; 316 . . . mass informationacquiring unit; 320 . . . article information registration unit; 330 . .. input/output unit; 340 . . . registration information generating unit;342 . . . coordinate information adding unit; 344 . . . holdinginformation adding unit; 346 . . . specification information addingunit; 350 . . . registration request sending unit; 410 . . .manipulator; 412 . . . picking hand; 414 . . . robot arm; 420 . . .driving unit; 430 . . . drive control unit; 510 . . . recognitioninformation storing unit; 520 . . . recognition unit; 530 . . . planningunit; 540 . . . execution unit; 550 . . . concurrent processing controlunit; 610 . . . position and posture information generating unit; 620 .. . point cloud information generating unit; 710 . . . holding planningunit; 720 . . . arrangement planning unit; 730 . . . evaluationinformation storing unit; 740 . . . trajectory candidate extractingunit; 750 . . . trajectory planning unit; 760 control informationgenerating unit; 810 . . . control information sending unit; 820 . . .execution monitoring unit; 1400 . . . data table; 1410 . . . work ID;1420 . . . order ID; 1430 . . . contents; 1432 . . . article ID; 1434 .. . supply pallet ID; 1436 . . . delivery pallet ID.

What is claimed is:
 1. A control device for controlling a pickingapparatus that arranges a work picked at a first position of a firstcontainer by using a picking hand at a second position of a secondcontainer being different from the first container, the control devicecomprising: a point cloud information acquiring unit for acquiring pointcloud information of a region including at least one of the picking handand the work at a time which is at least one of (i) when the pickinghand moves the work in the second container and (ii) when the pickinghand arranges the work at the second position; an abnormality detectionunit for detecting an abnormality based on the point cloud informationacquired by the point cloud information acquiring unit; and aninstruction sending unit for sending, to the picking apparatus, aninstruction for suspending an operation of the picking apparatus inresponse to the abnormality detection unit detecting the abnormality,wherein: the abnormality detection unit is for detecting a load collapseinside the second container based on the point cloud informationacquired by the point cloud information unit, and the instructionsending unit is for sending, to the picking apparatus, the instructionfor suspending the operation of the picking apparatus in response to theabnormality detection unit detecting the load collapse.
 2. The controldevice according to claim 1, further comprising an image acquiring unitfor acquiring image data of an image including at least one of thepicking hand and the work in a field angle at the at least one time,wherein: the point cloud information acquiring unit acquires the pointcloud information based on the image acquired by the image acquiringunit.
 3. The control device according to claim 1, wherein the pointcloud information is three-dimensional point cloud information.
 4. Thecontrol device according to claim 1, wherein: the abnormality detectionunit is for detecting the abnormality based on (i) the point cloudinformation and (ii) at least one selected from a group including: ameasurement value for a sensor arranged on the picking apparatus; acontrol value for at least one of the picking hand, and a robot armcoupled with the picking hand; and at least one of torque, a voltagevalue and a current value for a driving unit that drives at least one ofthe picking hand and the robot arm.
 5. The control device according toclaim 1, wherein: the abnormality detection unit is for detecting aninterference based on the point cloud information acquired by the pointcloud information unit, and the instruction sending unit is for sending,to the picking apparatus, an instruction for suspending an operation ofthe picking apparatus in response to the abnormality detection unitdetecting the interference.
 6. A tangible, non-transitorycomputer-readable medium having processor instructions stored thereonthat, when executed cause the processor to perform operations forcontrolling a picking apparatus that arranges a work picked at a firstposition of a first container by using a picking hand at a secondposition of a second container being different from the first container,the operations comprising: acquiring point cloud informationrepresenting a region including at least one of the picking hand and thework at a time that is at least one of (i) when the picking hand movesthe work in the second container, and (ii) when the picking handarranges the work at the second position; detecting an abnormality basedon the point cloud information acquired in the acquiring point cloudinformation and sending, to the picking apparatus, an instruction forsuspending an operation of the picking apparatus in response to thedetecting abnormality detecting the abnormality, wherein: the detectingthe abnormality includes detecting a load collapse inside the secondcontainer based on the point cloud information acquired in the acquiringpoint cloud information, the sending the instruction includes sending,to the picking apparatus, the instruction for suspending the operationof the picking apparatus in response to detecting the load collapse. 7.A control method for controlling a picking apparatus that arranges awork picked at a first position of a first container by using a pickinghand at a second position of a second container being different from thefirst container, the control method comprising: acquiring point cloudinformation of a region including at least one of the picking hand andthe work at a time that is at least one of (i) when the picking handmoves the work in the second container, and (ii) when the picking handarranges the work at the second position; and detecting an abnormalitybased on the point cloud information acquired in the acquiring pointcloud information and sending, to the picking apparatus, an instructionfor suspending an operation of the picking apparatus in response to thedetecting abnormality detecting the abnormality, wherein: the detectingthe abnormality includes detecting a load collapse inside the secondcontainer based on the point cloud information acquired in the acquiringpoint cloud information, the sending the instruction includes sending,to the picking apparatus, the instruction for suspending the operationof the picking apparatus in response to detecting the load collapse.